Vehicle drive assist system

ABSTRACT

A vehicle drive assist system comprises a camera for picking up an image of an area existing in an advancing direction of a vehicle; display means for displaying the image picked up by the camera; steering angle detecting means for detecting a steering angle for steering the vehicle; traveling path predicting means for predicting a traveling path of the vehicle on the basis of the steering angle detected by the steering angle detecting means; and drive assist means for overlaying on the display means drive assist information containing the vehicle predictive traveling path predicted by the traveling path predicting means and guide lines prolonged from the lines defining the width of the vehicle body on the image of the area existing in the vehicle advancing direction.

This is a Division of application Ser. No. 09/598,270 filed Jun. 21,2000. The disclosure of the prior application is hereby incorporated byreference herein in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a vehicle parking assist device forassisting a driver in his driving by use of an image obtained by takinga scene of an area around a vehicle.

2. Description of the Related Art

To drive a vehicle, e.g., automobile, in safety, the driver needs tolook traffic conditions around the vehicle attentively. The driverconstantly watches the environment of the vehicle and makes efforts tograsp environmental conditions of the vehicle, while driving thevehicle. When the grasping effort depends on only the sense, there is alimit in perfectly grasping the environmental conditions. The humanvisual sensation is very important in grasping the environmentalconditions. However, it is effective only in a direction in which thedriver looks. The driver must drive the vehicle under the condition thatareas that are shaded by the vehicle and from driver's view, i.e., thedead spaces, inevitably exist.

Major dead spaces as viewed from the driver are areas near the wheelsand just before and after the vehicle body. In the dead spaces near thewheels, the driver cannot check the orientations of the wheels per sewhose running directions change in accordance with a turn of a steeringwheel. The steering wheel may be turned by plural turns in onedirection. Accordingly, if the steering wheel is apparently at a neutralposition, it is unknown whether or not the wheels are oriented for astraight drive. Further, visibility is limited in the backward area ofthe vehicle, so that the driver takes much care when he backs thevehicle into a narrow space.

Particularly, the parking technique is one of the most difficult drivingtechniques when a normal driver drives a vehicle. In parking thevehicle, the driver must steer the vehicle so that the vehicle movesinto a narrow parking space without touching of his car with anothervehicle or an obstructive object or without being mired in a ditch. Theobstructive object and the other vehicle which the vehicle driven by thedriver may touch usually exist in an area existing within a dead anglewhen viewed from the driver. Accordingly, the driver must steer thevehicle while exactly grasping relative positional relationship of thevehicle body to other objects.

In the technique to park the vehicle, particularly, particular thetechnique to park the vehicle in a parking space between the adjacentvehicles of those being parked in a row in a back-to-front fashion (thisparking will be referred to as a “series parking”) is one of the mostdifficult driving techniques when a normal driver drives the vehicle. Inthe series parking, the driver must back the vehicle into a narrowparking space and reversely turns the steering wheel. Further, since thedriver backs the vehicle, a parking space in which the vehicle is to beparked is the dead space when viewed from the driver and the driverinsufficiently knows the environment around the parking space. In thisstate, the driver must parks the vehicle in the parking space

A drive assist technique in which when the vehicle is moved backward, itis disclosed in JP-A-1-14700 that an image of the backward or sidewardarea of the vehicle and a predictive traveling path curve are displayed.In the background technique, a predictive traveling path of the vehicle,which is predicted depending on a steering angle in the vehicle, is laidover an image of the backward area of the vehicle. Therefore, the drivercan check both the steering angle of the steering wheel and safety inthe driving in the backward area. According to the drive assistance ofJP-A-1-14700, a steering angle of a steering wheel is detected by asteering sensor, a predictive traveling path corresponding to a steeringangle of the steering wheel when the vehicle backs is overlaid on theimage picked up by the camera, and frames are depicted in a state thatthose are equidistantly arrayed along the predictive traveling path or asimplified vehicle is depicted, whereby the driver easily understands insense the predictive traveling path.

A drive assist technique which detects a length of a parking space byvarious sensors, and assists the driver in his parking by use of thedetection result, is disclosed in JP-A-7-44799. According toJP-A-7-44799, when the vehicle passes the side of a series parkingspace, the vehicle parking assist device measures the parking space, andcalculates an amount of steering operation, a time of depressing theaccelerator or a time of depressing a brake pedal by using the parkingspace information measured and vehicle information, and guides thedriver for the series parking.

A drive assist technique for assisting the driver in his driving by useof an image produced by picking up a scene around the vehicle isdisclosed in JP-A-10-283592, JP-A-10-257482, JP-A-6-255423, JP-A-239400,JP-A-5-143894, JP-A-7-192199, and JP-A-57-150044U.

According to JP-A-10-283592, based on three dimensional analysis of animage picked up by a camera, it is judges whether or not the selfvehicle touches an obstructive object, and the driver is informed of thejudgement result while at the same time displays a relation between theself vehicle and the obstructive object.

JP-A-10-257482 discloses the drive assist technique which seamlesslycomposes an image of the backward area of the vehicle, an image of theleft side of the vehicle, and an image of the right side of the same,and displays the composed image.

JP-A-6-255423 discloses a drive assist technique which displays an imagepicked up by a camera together with a distance scale for the driveassist.

JP-A-4-23940 discloses a drive assist technique in which a camera, whichmay be horizontally turned over a range of 360°, is attached to theceiling of the vehicle with the aid of a turntable, a scene in areasexisting in desired directions around the vehicle are picked up by thecamera, and the images thus picked up are converted and composed forease of watching by the driver, and the resultant image is used fordrive assist.

JP-A-5-14389 discloses a drive assist technique in which scenes of theareas in the front left, front and front right of the vehicle are pickedup by three cameras, and varies the segmental display areas or windowsof the screen on which the images from the three cameras are displayed,thereby effecting the drive assist.

JP-A-7-19219 discloses a drive assist technique in which when thevehicle passes though a narrow road, an obstructive object is detectedfrom an image produced by picking up a scene of the frontward area, anda gap between it and the obstructive object, thereby effecting the driveassist.

JP-A-57-150044U discloses a drive assist technique in which when anabnormality occurs, the contents of a display by a display device ischanged from the display contents for a normal mode to the displaycontents for a normal mode, thereby effecting the drive assist.

However, as in the drive assist technique of JP-A-7-44799, a length ofthe parking space is estimated on the basis of a detected distance, andthe drive assist is carried out for a garage parking and a seriesparking. In this technique, an image of the dead space for the driver isnot displayed for the drive assist. Therefore, the technique fails tocompletely remove an anxiety of the driver.

In the background art of the JP-A-64-4700, a three-dimensional displayalong a predictive traveling path is used for enhancing the illusion ofdepth of a predictive traveling path image. The technique to merelydisplay 3-dimensional images equidistantly arrayed along the travelingpath predicted on the basis of a steering angle or move them along thetraveling path is almost ineffective in checking a possibility that thevehicle will touch an obstructive object located near the predictivetraveling path, because the driver must judge whether or not a steeringangle of the steering wheel is proper.

Moreover, the information display device for displaying the images,usually installed to the vehicle, must be mounted in a narrow space nearthe driver seat. Therefore, the screen of the information display deviceis not large. For this reason, when the driver watches the displaycontaining the backward area and the predictive traveling path curve,and judges if the predictive traveling path is proper, it is difficultfor the driver to exactly judge if the predictive traveling path isproper.

Where the technique, as disclosed in JP-A-7-44799, to merely make anexact measurement of the parking space is used, it enables the drive tocheck if the parking space is sufficiently large, but it canunsatisfactorily assist the driver in the driving operation, which isnecessary for the advancing of the vehicle to the parking space. Thepublication of JP-A-7-44799 describes in detail the measurement of theparking space, and refers to only the instructions of the drivingoperations for parking that is given to the driver. However, specificinstructions are unclear in the description of the publication.

The parking assist technique, as disclosed in JP-A-10-283592, whichmerely judges if the self vehicle touches the obstructive object fromthe three-dimensional analysis of the image, fails to effect such aparking assist as guide the vehicle to a predetermined parking space.

On summarizing the above, in the drive assist technique in which theimage picked up by the camera is used for the drive assist, an image ofthe dead space for the driver is used for watching the dead space.Therefore, the driver may check the environment conditions around thevehicle by watching the image, and the driver may take a feeling ofsecurity. However, those related techniques do not always provide properdrive guide information having a simple construction.

SUMMARY OF THE INVENTION

Accordingly, an object of the present invention to provide a vehicleparking assist device which assists the driver in his driving, using asimple image display or simple voice system.

According to the present invention, there is provided a vehicle driveassist system comprising;

a camera for picking up an image of an area existing in the advancingdirection of a vehicle;

display means for displaying an image picked up by the camera;

steering angle detecting means for detecting a steering angle forsteering a vehicle;

traveling path predicting means for predicting a traveling path of avehicle on the basis of a steering angle detected by the steering angledetecting means; and

drive assist means for overlaying drive assist information containing avehicle predictive traveling path predicted by the traveling pathpredicting means and guide lines prolonged from the lines defining thewidth of the vehicle body on the image of the area existing in thevehicle advancing direction, which is displayed by the display means.

In the invention, a vehicle predictive traveling path predicted on thebasis of a steering angle of the vehicle and guide lines prolonged fromthe lines defining the width of the vehicle body are overlaid on theimage of the area existing in the vehicle advancing direction, which isdisplayed by the display means, for example. Therefore, the vehicleparking assist device appropriately compensates for the lack of thevisual sensation in the vehicle drive.

In the vehicle drive assist device, the drive assist means selects adisplay of the predictive traveling path and/or guide lines.

The invention selects a display of the predictive traveling path and/orguide lines. Therefore, the image from the camera and the predictivetraveling path curve or one of the guide lines, or that image, thepredictive traveling path curve and the guide lines are displayed,thereby performing a proper drive assist.

In the vehicle parking assist device, the drive assist means varies abrightness of a display of the predictive traveling path and/or guidelines in accordance with a brightness of the environment.

In the invention, a brightness of a display of the predictive travelingpath and/or guide lines may be varied in accordance with a brightness ofthe environment.

In the vehicle drive assist device, the drive assist means varies acolor arrangement of a display of the predictive traveling path and/orguide lines in accordance with a color arrangement of the environment.

In the invention, a color arrangement of a display of the predictivetraveling path is varied in accordance with a color arrangement of theenvironment. Therefore, an easy-to-understand guide display is presentedto the driver, whereby a proper drive assist is performed.

In the vehicle drive assist device, the vehicle includes a brightnesssensor for sensing a brightness of the environment to select anillumination system, and the drive assist means varies a brightness or acolor arrangement of a display of the predictive traveling path and/orguide lines in accordance with a brightness sensed by the brightnesssensor.

In the invention, a brightness sensor attached to the vehicle senses abrightness of the environment, and a brightness or a color arrangementof a display of the predictive traveling path and/or guide lines isvaried in accordance with a color arrangement of the environment.Therefore, a display which is easy for the driver to understand isautomatically performed.

In the vehicle drive assist device, the drive assist means varies acolor arrangement of a display of the predictive traveling path and/orguide lines in accordance with a background color of an image picked upby the camera.

In the invention, a color arrangement of a display of the predictivetraveling path and/or guide lines is varied in accordance with abackground color of an image picked up by the camera. With this feature,when the background color is bright, the predictive traveling path curveand the guide lines may be displayed using a dark color. When thebackground color is dark, the latter displayed may be bright in color.Therefore, an easy-to-understand drive assist may be presented to thedriver.

In the vehicle drive assist device, said drive assist means includesparking drive judging means for judging whether or not the drive of thevehicle is a parking drive,

parking end judging means arranged such that when said parking drivejudging means judges that the vehicle drive is a parking drive, saidparking end judging means judges whether or not the parking drive ends,

storing means arranged such that when said parking end judging meansjudges that the parking drive ends, said storing means stores a selectcondition for displaying said predictive traveling path and/or guidelines, and

select means arranged such that when said parking drive judging meansjudges that the vehicle drive is a parking drive, said select meansselects a display condition of said drive assist information inaccordance with the contents of said storing means.

In the invention, the final display condition at the end of the parkingassist is stored, and when the next parking drive is performed, thedisplay condition is changed to the stored one. Therefore, there iseliminated the work by the driver to change the display condition toanother. The parking assist is carried out with a display which is easyfor the driver to understand.

In the vehicle drive assist device, said camera picks up an image by awide angle field, and said drive assist displays said guide lines curvedin accordance with a distortion of a wide angle image picked up by saidcamera.

In the invention, said guide lines, when displayed, are curved inaccordance with a distortion of a wide angle image picked up by saidcamera. Therefore, the guide lines exactly correspond to the linesrespectively prolonged from the lines defining the width of the vehiclebody.

In the vehicle drive assist device, the drive assist means displays thepredictive traveling path and/or guide lines except the portions thereofnear the vehicle.

In the invention, in displaying the predictive traveling path curve andthe guide lines, the vehicle body is present on this side of them in theregion near the vehicle. Therefore, the width is not coincident with theback width of the vehicle apparently. In the invention, thenon-coincident portion is not displayed, and the driving for thisportion depends on the driver's judgement, whereby an unnatural displayis avoided.

In the vehicle drive assist device, the drive assist means displays thepredictive traveling path and/or guide lines such that the portionsthereof near the vehicle are broadened.

In the invention, portions of the predictive traveling path and/or guidelines, which are near the vehicle, are broadened. Therefore, a safety issecured.

In the vehicle drive assist device, the drive assist means displays aline indicative of a stop position of the vehicle at a position on thepredictive traveling path and/or guide lines a line indicative of a stopposition of the vehicle, which is located near the vehicle.

In the invention, a line indicative of a stop position of the vehicle isdisplayed at a position on the predictive traveling path or the guidelines, which is located near the vehicle. Therefore, an easy parkingdrive assist is secured in a manner that when the moving vehicle framereaches the stop line, the vehicle is stopped.

In the vehicle drive assist device, the drive assist displays the guidelines when braking is effected.

In the invention, when the driver operates the brake, the guide linesare displayed. Therefore, when the operator operates the brake andchecks if the predictive traveling path is parallel to the guide lines,and when the traveling path is parallel to the guide lines, the steeringwheel is at the center position.

In the vehicle drive assist device, the drive assist displays the guidelines in the form of dotted lines.

In the invention, the guide lines are displayed by dotted lines.Therefore, the guide lines may be made unnoticeable. When the predictivetraveling path is additionally displayed, the predictive traveling pathis intensified in brightness on display. Accordingly, the drive guidewith a marked display of the predictive traveling path is presented tothe driver.

The vehicle drive assist device further comprises an obstacle sensor fordetecting presence or absence of an obstructive object on a vehicletraveling path, and wherein the drive assist varies a length of theguide lines in accordance with a distance ranging to an obstructiveobject detected by the obstacle sensor.

In the invention, a length of the guide lines is varied in accordancewith a distance ranging to an obstructive object detected by theobstacle sensor, such as a back sonar. Therefore, the display gives thedriver a drive feeling resembling an actual driving feeling. A driverassist which is easy for the driver to understand is provided.

In the vehicle drive assist device, the drive assist means displays theguide lines in a state that the width of the guide lines, whendisplayed, is wider than the width of the vehicle.

In the invention, the width of the guide lines displayed is wider thanthe width of the vehicle. Therefore, the drive assist is carried outwith an easy-to-handle operation. For example, when one of the guidelines lies on one side of the parking frame, the driver returns thesteering wheel to the center position.

According to the present invention, there is provided a vehicle driveassist system comprising;

a camera for picking up an image of an area existing in the advancingdirection of a vehicle; display means for displaying an image picked upby the camera;

reverse gear mode detecting means for detecting as to whether or not atransmission of the vehicle is set to a reverse gear mode; and

drive assist means arranged such that when the reverse gear modedetecting means judges that the transmission is set to the reverse gearmode, the drive assist means changes a display by the display means andinforms the driver that the transmission is in a reverse gear mode,thereby effecting the drive assist.

When the transmission is set to the reverse gear mode, a display by thedisplay means changes. The driver is informed of the transmission beingin a reverse gear mode. Therefore, an accident which will occur when agear position of the transmission has slipped from the driver's memorycan be prevented.

According to the present invention, there is provided a vehicle driveassist system comprising;

a camera for picking up an image of an area existing in the advancingdirection of a vehicle;

display means for displaying an image picked up by the camera;

an obstacle sensor for detecting presence or absence of an obstructiveobject in a backward area of the vehicle;

drive assist means arranged such that when an obstructive object isdetected by the obstacle sensor, the drive assist means informs thedriver of presence of an obstructive object by changing a display by thedisplay means.

In the invention, when an obstructive object is detected by the obstaclesensor such as a back sonar or corner sensors, a display by the displaymeans is changed to inform the driver of the presence of the obstructiveobject. The display calls the driver's attention to the obstructiveobject, thereby preventing the vehicle from touching the obstructiveobject.

In the vehicle drive assist device, the drive assist means displaysinformation in the form of character telop for the drive assist.

In the invention, the drive assist information is displayed in the formof character telop. Therefore, driver's attention is reliably called toavoid a dangerous accident.

In the vehicle drive assist device, the drive assist means displaysinformation in the form of a color change of the entire display.

Thus, driver's attention is attracted by a color change of the entiredisplay.

According to the present invention, there is provided a vehicle driveassist system comprising;

a camera for picking up an image of an area existing in the advancingdirection of a vehicle; display means for displaying an image picked upby the camera;

drive assist means arranged such that the drive assist means detects adrive condition and judges whether or not the drive mode is a parkingdrive mode, and when the drive mode is a parking drive mode, the driveassist means causes said display means to display an image display ofthe display means and an image display for parking drive assist; and

safety check means for displaying a telop to urge the driver to make asafety check on the image for the parking drive assist by the driveassist means.

In the invention, when the drive mode is a parking drive mode, the imagefor the parking assist and a telop for urging the driver to make asafety check by the eye are displayed. Therefore, it is prevented thatthe driver is riveted to the display by the display means.

According to the present invention, there is provided a vehicle parkingassist device comprising:

a camera for picking up an image of a backward area of a vehicle;

display means for displaying an image picked up by the camera;

obstacle sensor for detecting presence or absence of an obstacle in thebackward area of the vehicle;

drive assist means arranged such that the drive assist detects a drivecondition and checks if the vehicle backs for parking, and when thevehicle backs, the drive assist means displays drive assist informationfor parking; and

safety check means arranged such that the safety check means checks theparking drive ends on the basis of the result of the obstacle detectionby the obstacle sensor, and when the parking drive ends, the safetycheck means causes the display means to display the end of the parkingdrive.

In the invention, the safety check means checks the parking drive endson the basis of the result of the obstacle detection by the obstaclesensor, such as a back sonar, and when the parking drive ends, a displayof the end of the parking drive is presented on the screen of thedisplay means.

According to the present invention, there is provided a vehicle parkingassist device comprising:

a camera for picking up an image of a backward area of a vehicle;

display means for displaying an image picked up by the camera;

reverse gear mode detecting means for judging transmission of thevehicle is set to a reverse gear mode; and

drive assist means arranged such that when the reverse gear modedetecting means judges that the transmission is set to the reverse gearmode, the drive assist means limits a speed of the vehicle to the presetupper limit of the vehicle speed or lower, thereby effecting the driveassist.

In the invention, when the vehicle is moving backward, a speed of thevehicle is limited to the preset upper limit of the vehicle speed orlower.

Therefore, collision of the vehicle when it is moved backward isprevented or shock when the collision occurs is lessened.

According to the present invention, there is provided a vehicle parkingassist device comprising:

a camera for picking up an image of a backward area of a vehicle;

display means for displaying an image picked up by the camera;

obstacle sensor for sensing presence or absence of an obstacle in thebackward area of the vehicle; and drive assist means arranged such thatwhen the obstacle sensor detects an obstacle, the drive assist meanslimits a speed of the vehicle to the preset upper limit of the vehiclespeed or lower, thereby effecting the drive assist.

In the invention, when the obstacle sensor detects an obstacle such as aback soar, the drive assist means limits a speed of the vehicle to thepreset upper limit of the vehicle speed or lower. Therefore, collisionof the vehicle when it is moved backward is prevented or shock when thecollision occurs is lessened.

According to the present invention, there is provided an vehicle parkingassist device comprising:

a camera for picking up an image of a backward area of a vehicle;

display means for displaying an image picked up by said camera;

reverse gear mode detecting means for judging transmission of thevehicle is set to a reverse gear mode; and drive assist means arrangedsuch that when said reverse gear mode detecting means judges that thetransmission is set to the reverse gear mode, said drive assist meanschanges an illumination direction of a lamp illuminating a backward areaof the vehicle, thereby effecting the drive assist.

When the transmission is set to the reverse gear mode, the drive assistmeans changes an illumination direction of a lamp illuminating abackward area of the vehicle, thereby effecting the drive assist.Therefore, even when the parking lot is dark at night, for example, theimage picked up by the camera is increased in brightness, and hence aneasy-to-understand drive assist is secured.

A vehicle drive assist device, wherein the lamp is optionally installed.

In the invention, the lamp for illuminating a visual field of the camerawhen the vehicle backs is optionally attached to the device. Therefore,good visibility is secured in the parking drive at night, for example.

In the vehicle drive assist device, the vehicle is equipped with acar-carried device including a navigation device and/or audio device,and further the device comprises select means for causing the displaymeans to selectively display the image from the camera and the imagefrom the drive assist means or the image from the car-carried device.

In the invention, the image from the camera and the image from the driveassist means or the image from the car-carried device, e.g., anavigation device or an audio device, may be selectively displayed whenrequired in the driving.

According to the present invention, there is provided a vehicle parkingassist device comprising:

a camera for picking up an image of an area existing in a vehicle;

display means for displaying an image picked up by the camera; and

drive assist means for switching drive assist information for parkingoverlaid on an image displayed by the display means between the driveassist information of a normal parking mode and that of a series parkingmode.

Drive assist means for switching drive assist information for parkingmay be switched between the drive assist information of a normal parkingmode and that of a series parking mode in accordance with a normalparking or a series parking.

In the vehicle drive assist device, the drive assist means selects thedrive assist information of a series parking mode when the turning on ofa winker lamp or a hazard lamp and the shifting of the transmission tothe reverse gear mode are simultaneously performed.

In the invention, a display mode of the display means is changed to aseries parking mode in which the display means displays the drive assistinformation for the series parking, in accordance with a lightingcondition of the winker lamp or hazard lamp.

In the vehicle drive assist device, the vehicle is equipped with anavigation device, and the drive assist means arranged such that thedrive assist means judges whether a parking position in which thevehicle is to be parked is in a parking lot or on a road, on the basisof the present position of the vehicle derived from the navigationdevice, and when the parking position is in the parking lot, the driveassist means selects the drive assist information of the normal parkingmode, and when the parking position is on the road, the drive assistmeans selects the drive assist information of the series parking mode.

In the invention, the drive assist means judges whether a parkingposition in which the vehicle is to be parked is in a parking lot or ona road, on the basis of the present position of the vehicle derived fromthe navigation device. When the parking position is in the parking lot,the normal parking mode is selected, and

when the parking position is on the road, the series parking mode isselected.

In the vehicle drive assist device, the drive assist means selects thedrive assist information of the normal parking mode or the seriesparking mode in accordance with a change of a steering angle forsteering the vehicle.

In the invention, the normal parking mode or the series parking mode isselected in accordance with a change of a steering angle for steeringthe vehicle.

Therefore, the series parking including a reverse turn of the steeringwheel or the normal mode not including the steering reverse turn may beautomatically selected.

According to the present invention, there is provided a vehicle parkingassist device comprising:

a plurality of cameras being disposed such that the photographing areascovered by the cameras are shifted so as to pick up a scene around thevehicle;

display means for displaying images picked up by the cameras; and

drive assist means arranged such that a display screen of the displaymeans is segmented into plurality of display areas, and the images bythe plurality of cameras are simultaneously displayed on those segmentaldisplay areas, thereby effecting the drive assist.

In the invention, a plurality of cameras are disposed such that thephotographing areas covered by the cameras are shifted so as to pick upa scene around the vehicle. A display screen of the display means issegmented into plurality of segmental display areas in accordance withthe arrangement of the cameras. The images by the plurality of camerasare simultaneously displayed on those segmental display areas.Therefore, the driver may look a plurality of images on the same screenof the display means. Accordingly, the driver may drive the vehiclewhile watching the images of the broad angular expanse.

In the vehicle drive assist, further comprising drive conditiondetecting means for detecting a vehicle drive condition, and wherein thedrive assist means selectively changes a display in accordance with adrive condition detected by the drive condition detecting means.

In the invention, a display on the screen is changed in accordance witha drive condition. Accordingly, the drive assist is carried out by useof a display of visual information which is easily understood by thedriver since an image display by the display means is automaticallychanged in accordance with vehicle running conditions. In a specificexample, in the normal running, two images of the front left and rightareas of the vehicle are displayed, and in the backward running, threeimages of the front left and right areas, and the backward area aredisplayed.

The vehicle drive assist device further comprises drive conditiondetecting means for detecting a vehicle drive condition, and wherein thedrive assist means selectively changes a display in accordance with adrive condition detected by the drive condition detecting means.

In the invention, the vehicle is displayed in a simplified form on thescreen of the display means.

Accordingly, a relationship of the image relative to the vehicle isdisplayed in an easy-to-understand manner on the basis of the displaysof the vehicle an the image, whereby a proper drive assist is providedto the driver.

In the vehicle drive assist device, the drive assist means moves adisplay position of the vehicle according to an image from the camera.

Since a display position of the vehicle is moved in accordance with animage from the camera, a change of the image display is easy tounderstand.

In the vehicle drive assist device, the drive assist means operates todisplay the directions and photographing areas covered by the cameraswhen the vehicle is displayed in a simplified form.

In the invention, the vehicle is displayed in a simplified form, and thedirections and photographing areas covered by the cameras are alsodisplayed.

Therefore, the driver easily recognizes which camera picked up the imagebeing displayed.

In the vehicle drive assist device, the vehicle is equipped with acar-carried device including a navigation device and/or audio device,and drive assist means arranged such that a display screen of thedisplay means is segmented into plurality of display areas, and theimages from the plurality of cameras and an image from the car-carrieddevice are displayed on those segmental display areas, thereby effectingthe drive assist.

In the invention, the driver may look both the image by each camera andthe image by the navigation device or the audio device on the samedisplay screen.

In a vehicle drive assist device, drive assist means changes thecameras, which causes the display means to display, one to another atpredetermined time intervals.

In the invention, the images by the plurality of cameras are changed oneto another at predetermined time intervals, to thereby eliminate themanual work of the driver to change the images.

In the vehicle drive assist device, the drive assist means uses a wipeto change the images.

In the invention, the wipe is used for changing the images by theplurality of cameras one to another. The shifting of one image toanother image is continuous. Further, the changing of the images isautomatically performed without the manual operation by the driver.

In the vehicle drive assist device, a camera for picking up a scene of aside area of the vehicle is included in the cameras, and the driveassist means operates to display a side image concerning a directionindicator, which is operated being interconnected with an operation ofthe direction indicator.

In the invention, the image of the area existing in the vehicleadvancing direction may be displayed being interconnected with thedirection operation. Accordingly, the vehicle parking assist deviceeffectively assists the driver to make a safety check.

In the vehicle drive assist device, the drive assist means changes theimages one to another in accordance with a steering angle of turn of asteering wheel for steering the vehicle.

In the invention, the images are changed one to another in accordancewith a steering angle of turn of a steering wheel. Therefore, the imageof the area existing in the vehicle advancing direction is automaticallydisplayed, and the device assists the driver safely and effectively.

In the vehicle drive assist device, the drive assist means changes acondition of segmenting the screen of the display means in accordancewith a steering angle of the steering wheel.

In the invention, the screen segmenting condition is changed inaccordance with a steering angle of the steering wheel. Therefore, theimage of the area located in the advancing direction is displayed on awide display screen, and a safety check is easy in the drive assist.

The vehicle drive assist device further includes comprising an obstaclesensor for detecting presence or absence of an obstructive object on avehicle traveling path, and wherein when the obstacle sensor detects anobstacle, the drive assist means causes the display means to display asensing direction of the obstacle sensor.

In the invention, wherein when the obstacle sensor detects an obstacle,a sensing direction of the obstacle sensor is displayed. Therefore, thedevice informs the driver of a relation of the vehicle to theobstructive object on the display screen to attract the driver'sattention.

In the vehicle drive assist device, when the obstacle sensor detects anobstacle, the drive assist means causes the display means to display animage from the camera which picks up a sensing direction of the obstaclesensor.

In the invention, when the obstacle sensor, e.g., a corner sensor,detects an obstacle, an image including the obstacle is displayed. Thevehicle parking assist device calls driver's attention to the presenceof the obstacle and its state.

In the vehicle drive assist device, when the obstacle sensor detects anobstacle, the drive assist means causes the display means to shift adisplay position of an image, which comes from the camera which picks upa sensing direction of the obstacle sensor, to the center of the screenof the display means or to expand the display.

In the invention, while being interconnected with the obstacle sensor,e.g., a corner sensor, a display position of an image, which comes fromthe camera which picks up a sensing direction of the obstacle sensor, isshifted to the center of the screen of the display means or the displayis expanded. Therefore, the presence of the obstacle and its state aredisplayed in an ease-to-understand manner.

In the vehicle drive assist device, the drive assist means causes thedisplay means to overlay an alarm on a display of the sensing directionof the obstacle sensor.

In the invention, while being interconnected with the obstacle sensor,such as a corner sensor, an alarm is overlaid on a display of thesensing direction of the obstacle sensor. Therefore, the driver isintelligibly informed of the direction in which the obstacle iscontained, which is easy to understand.

In the vehicle drive assist device, when the images of the camerasadjacently disposed are simultaneously displayed, the drive assist meansadjusts the display areas and positions of those images so that thoseimages are continuously connected together.

In the invention, the images by the cameras adjacently disposed arecontinuously connected together to form a single composed image. Withthe composed image, the image covering an expanse of area, which is easyfor the driver to understand, is provided to the driver, to therebysupport the driving.

In the vehicle drive assist device, the camera are disposed so as totake scenes of the front, front left, front right, rear left and rearright areas of the vehicle.

The vehicle parking assist device of the invention is capable ofphotographing the entire circumferential area around the vehicle.

In the vehicle drive assist device, the drive assist means segments thescreen of the display means into the upper and lower display areas, andcauses the display means to display an image of the front of the vehicleon the central area of the upper display area of the screen, the frontleft and right images on the left and right display areas of the upperdisplay area, and an image of the rear area of the vehicle on thecentral area of the lower display area of the screen, the rear left andright images on the left and right areas of the lower display area ofthe screen of the display means.

The vehicle parking assist device of the invention is capable of makinga safety check on the entire circumferential area around the vehicle onone display.

In the vehicle drive assist device, the drive assist means causes thedisplay means to display an image selected from images picked up by thecameras disposed described above.

Only the image, which is selected, as needed, from images picked up bythe cameras so disposed as to pick up the entire circumferential areaaround the vehicle, is displayed. Accordingly, the display means isefficiently utilized to display a necessary image.

In the vehicle drive assist device, the drive assist means causes thedisplay means to display the images output from the cameras having thephotographing areas successively disposed in scrolling manner.

The images output from the cameras having the photographing areassuccessively disposed are scrolled into a single image. Therefore, thedriver is able to readily check the image of a broad area around thevehicle.

In the vehicle drive assist device, the display means includes a touchpanel, and the drive assist means changes an image display conditionaccording to an input to the touch panel.

The display means for displaying an image picked up by a plurality ofcameras is equipped with a touch panel. Therefore, the driver may changea display condition of an image by merely touching the touch panel.

In the vehicle drive assist device, according to an input to the touchpanel, the drive assist means displays an image at a position as inputin a state that a time axis of its display is extended, and after apredetermined time elapses, the drive assist means returns the extendedtime axis to the original one.

In the invention, when the driver touches the touch panel to enlarge theimage, the image resumes its original size after a predetermined timeelapses. Therefore, the manual operation work is lessened to provide aneffective drive assist.

According to the present invention, there is provided a vehicle parkingassist device comprising:

a camera for picking up a scene in a frontward area of a vehicle;

storing means for reading an image picked up by the camera into thestoring means per se; and

drive assist means for acquiring information for vehicle drive assist onthe basis of the image stored in the storing means.

In the invention, a camera picks up a scene in a frontward area of avehicle, and storing means reads an image picked up by the camera intothe storing means per se. Since the image is stored into the storingmeans in the form of vehicle drive assist information, the driver canreadily extract the image as information effective for drive assist. Thesame information may be transmitted to other devices, and theinformation may be used as the information for congestion in broadareas.

The vehicle drive assist device further comprises display means fordisplaying an image coming from the camera.

Since an image coming from the camera is displayed by the display means,the driver can check a frontward area of the vehicle by the displaymeans.

The vehicle drive assist device further comprises vehicle detectingmeans for detecting a vehicle forcibly running into a place in front ofa self vehicle by processing the image coming from said camera.

In the invention, a vehicle forcibly running into a place in front of aself vehicle is detected by processing the image coming from saidcamera. Therefore, the vehicle drive assist device effectively effectsthe drive assist of, for example, measuring a distance from the selfvehicle to the preceding vehicle.

According to the invention, there is provided a vehicle parking assistdevice comprising:

a camera, attached to the top of a vehicle, for picking up a scenearound a vehicle as viewed from the top of the vehicle;

display means for displaying the image picked up by the camera; and

drive assist means arranged such that the drive assist means detect alimit of a vehicle height from the image picked up by the camera, andinforms a driver of the detection result as drive assist information.

In the invention, a camera is attached to the top of a vehicle and picksup a scene around a vehicle as viewed from the top of the vehicle.Display means displays the image picked up by the camera. A limit of avehicle height is detected from the image picked up by the camera.Therefore, the driver can readily check whether or not the vehicle maypass under an elevated bridge, for example.

According to the invention, there is provided a vehicle parking assistdevice comprising:

a camera for picking up scenes in the frontward and backward areas of avehicle;

display means for displaying the image picked up by the camera;

running condition detecting means for detecting running condition of thevehicle; and

drive assist means for carrying out drive assist in accordance withrunning conditions detected by the running condition detecting means insuch a manner that in a normal running, the display means displays animage of the frontward area of the vehicle, and in a backward running,the display means displays more brightly an image of the backward areathan an image of the forward area.

In the invention, the display means displays an image of the frontwardarea of the vehicle in a normal running. In a backward running, thedisplay means displays more brightly an image of the backward area thanan image of the forward area. Therefore, the driver can watch thebackward area in detail when backing the vehicle. In this respect, aneffective drive assist is secured.

According to the present invention, there is provided a vehicle parkingassist device comprising:

a camera for picking up scenes in the frontward and backward areas of avehicle;

display means for displaying the image picked up by the camera;

running condition detecting means for detecting running condition of thevehicle; and

drive assist means; and

drive assist means for carrying out drive assist in accordance withrunning conditions detected by the running state detecting means by useof such a display that in a normal running, the display means displaysan image of the frontward area of the vehicle, and in a backwardrunning, the display means displays more brightly an image of thebackward area than an image of the forward area, in place of a displayframe for back mirrors.

In the invention, an image of the backward area is displayed in anintensifying manner, in place of a display frame for back mirrors.Accordingly, the image of the backward area is displayed in a displaymode familiar to the driver. The drive assist is carried out such thatthe driver can easily understand that the displayed image is the imageof the backward area.

In the vehicle drive assist device, the drive assist means modifies abackward area image displayed in the display frame for the back mirrorsinto an image having a field angle equal to that of the back mirror, andcauses the display means to display the image modified in field angle.

Where the camera has a wide field angle, the image picked up by thecamera is distorted, and as a result, it is difficult to grasp the senseof distance. To cope with this, the image is modified, by the cuttingand enlarging or reducing of the image, into an image having a fieldangle equal to that of the back mirror, whereby the displayed imagematches the sight feeling of the driver.

According to the present invention, there is provided a vehicle driveassist device comprising:

a camera, mounted on a vehicle, for picking up a scene of an areaexisting in the advancing direction of the vehicle;

a steering angle sensor for detecting a steering angle of the steeringwheel of the vehicle;

traveling path predicting means for calculating a predictive travelingpath curve depending on a steering angle detected by the steering anglesensor;

drive assist means for assisting the driver in his driving operation byadding a 3-dimensional information image containing a height of avehicle along the predictive traveling path curve derived from thetraveling path predicting means;

display means for displaying an image output from the camera and animage output the drive assist means; and

an obstacle sensor for detecting an obstructive object existing in thevehicle advancing direction;

wherein the drive assist means changes a display of the 3-dimensionalinformation image at a position on the predictive traveling path curve,which is close to a position where an obstructive object is detected bythe obstacle sensor, to a display which is different from displays ofthe 3-dimensional image at other positions.

In the vehicle drive assist device thus constructed, a scene of an areaexisting in the advancing direction of the vehicle is picked up by acamera mounted on a vehicle. A steering angle of the steering wheel ofthe vehicle is detected by a steering angle sensor, and traveling pathpredicting means calculates a predictive traveling path curve dependingon a steering angle detected by the steering angle sensor. Drive assistmeans assists the driver in his driving operation by adding a3-dimensional information image containing a height of a vehicle alongthe predictive traveling path curve. An image output from the camera andan image output the drive assist means are displayed by the displaymeans. An obstructive object existing in the vehicle advancing directionis detected by an obstacle sensor. At a position close to a positionwhere an obstructive object is detected, the drive assist means changesa display of the 3-dimensional information image to a display which isdifferent from displays of the 3-dimensional image at other positions.Therefore, the driver readily checks a possibility that the vehicle willtouch an obstructive object at a position close to the object.

In the invention, the drive assist means displays the 3-dimensionalinformation image in the form of planes each having a predeterminedshape which are arrayed at fixed distances along the predictivetraveling path curve.

In the vehicle drive assist device, the drive assist means displays the3-dimensional information image in the form of a configurationrepresenting a height of the vehicle, e.g., rectangular planes orvehicle projective planes, which are arrayed at fixed distances alongthe predictive traveling path curve. Therefore, the vehicle height isdisplayed in an ease-to-understand fashion. Further, a three-dimensionalcheck of a possibility that his vehicle will touch an obstructive objectis readily effected since at a position near the obstructive object, thedisplay is changed from that at other positions.

In the invention, the drive assist moves a plane of a predeterminedshape, which represents a height of the vehicle, along the predictivetraveling path curve.

In the vehicle drive assist device, the drive assist moves a plane of apredetermined shape, which represents a height of the vehicle, such asrectangular planes or vehicle projective planes, along the predictivetraveling path curve. Therefore, the driver can readily check apossibility that his vehicle will touch an obstructive object on thepredictive traveling path curve, while recognizing the vehicle height.

In the invention, the drive assist stops the plane of a predeterminedshape, which represents a height of the vehicle, at a position which isclose to a position where the obstructive object is detected.

In the vehicle drive assist device just mentioned, the drive assistmoves the plane of a predetermined shape, which represents a height ofthe vehicle, along the predictive traveling path curve, and stops theplane at a position which is close to a position where the obstructiveobject is detected. Therefore, the driver can readily check a positionclose to the obstructive object and a possibility that the vehicle willtouch the obstructive object.

In the invention, the drive assist changes a color of the 3-dimensionalinformation image at a position which is close to a position where theobstructive object is detected.

In the vehicle drive assist device, a color of the 3-dimensionalinformation image is changed at a position which is close to a positionwhere the obstructive object is detected. Therefore, the driver canreadily recognize from a color change that a display position of the3-dimensional image approaches to the obstructive object. Therefore, thedrive assist visually presents a possibility that the vehicle will touchan obstructive object, and a touching position if it touches the object,in an ease-to-understand manner.

In the invention, the drive assist means displays the 3-dimentionalinformation image in the form of a simulation image in which a selfvehicle moves in the image picked up by the camera.

In the vehicle drive assist device, a 3-dimensional simulation image ofthe moving vehicle is overlaid on the image picked up by the camera.Therefore, the driver recognizes how the vehicle moves, on the displayedimage, and hence he can check a possibility that the vehicle will touchthe obstructive object, in a simulation domain.

According to the present invention, there is provided a vehicle parkingassist device for assisting a driver in his parking operationcomprising:

a camera for picking up a scene of an area existing in an advancingdirection of a vehicle;

parking position detecting means for detecting a target parking positionin which the vehicle is to be parked, from the image picked up by thecamera;

steering angle calculating means for calculating a steering anglenecessary for the moving of the vehicle to the target parking positiondetected by the parking position detecting means;

steering angle detecting means for detecting a steering angle of thevehicle; and

parking assist means for presenting a steering operation guide to thedriver on the basis of the result of comparing a steering angle detectedby the steering angle detecting means with a steering angle calculatedby the steering angle calculating means.

In the thus arranged vehicle parking assist device, when the vehicle isparked, a scene of an area existing in an advancing direction of avehicle is picked up by a camera, and a target parking position in whichthe vehicle is to be parked, from the image picked up by the camera isdetected by parking position detecting means. A steering angle necessaryfor the moving of the vehicle to the target parking position detected bythe parking position detecting means, is calculated by steering anglecalculating means. A steering angle of the vehicle is detected bysteering angle detecting means. A steering angle detected by thesteering angle detecting means is compared with a steering anglecalculated by the steering angle calculating means by parking assistmeans. A steering operation guide is presented to the driver on thebasis of the result of the comparison. The driver is guided so as tomove the vehicle at the steering angle necessary for moving the vehicleto the target parking position. Therefore, the driver easily judges ifthe present steering angle is proper. Accordingly, if the driver steersaccording to the guide, the vehicle may be properly parked in the targetparking position.

The vehicle parking assist device further comprises: traveling pathpredicting means for calculating, depending on a steering angle, apredictive traveling path curve along which the vehicle will travel; andtouch judging means for judging as to whether or not the vehicletraveling along the predictive traveling path curve calculated by thetraveling path predicting means will touch an obstructive object. In thevehicle parking assist device thus constructed, when a steering anglecalculated by the steering angle calculating means is out of a range ofsteering angles within which a steering wheel may be turned for steeringor when the touch judging means judges that the vehicle traveling alongthe predictive traveling path curve, which is calculated by thetraveling path predicting means depending on the steering angle, willtouch the obstructive object, the parking assist means urges the driverto reverse the advancing direction of the vehicle.

In the vehicle parking assist device, traveling path predicting meanscalculates, depending on a steering angle, a predictive traveling pathcurve along which the vehicle will travel, and touch judging meansjudges as to whether or not the vehicle traveling along the predictivetraveling path curve calculated by the traveling path predicting meanswill touch an obstructive object. When the touch judging means judgesthat the vehicle traveling along the predictive traveling path curve, orwhen a steering angle calculated by the steering angle calculating meansis out of a range of steering angles within which a steering wheel maybe turned for steering, the parking assist means urges the driver toreverse the advancing direction of the vehicle. Thus, the vehicleparking assist device supplies to the driver information of the parkingbeing impossible and a timing necessary for the reverse turn of thesteering wheel. Accordingly, a guide is made so that the driver canproperly park the vehicle into the parking space.

According to the present invention, there is provided a vehicle parkingassist device for assisting a driver in his parking operationcomprising:

a camera for picking up an image of an area existing in an advancingdirection of a vehicle;

parking space recognizing means for detecting a parking space in whichthe vehicle is to be parked, from the image picked up by the camera; and

parking assist means for judging as to if a parking stopper for blockingthe advancing of the vehicle exists in the parking space recognized bythe parking space recognizing means and for informing the driver of theresult of the judgement.

The camera picks up a scene of an area existing in an advancingdirection of a vehicle, and the parking space recognizing means detectsa parking space in which the vehicle is to be parked, from the imagepicked up by the camera. The parking assist means judges as to if aparking stopper for blocking the advancing of the vehicle exists in theparking space recognized by the parking space recognizing means andinforms the driver of the result of the judgement. Therefore, the drivercan know the presence or absence of the parking stopper. When he stopsthe vehicle at a parking position, he can steer and brake whilereferring to the stopper.

The vehicle parking assist device further comprises voice guide meansfor carrying out a guide by the parking assist by voice.

In the vehicle parking assist device, the parking assist guide ispresented to the driver by voice by the voice guide means. Therefore,the driver may take a drive guide necessary for the parking by the senseof hearing, not the sense of sight.

The voice guide means also gives the driver warning to urge the driverto watch environmental conditions around the vehicle.

In the vehicle parking assist device, the voice guide means urges thedriver to watch the environment around the vehicle in the form of voicewarning to the driver. Therefore, sufficient driver's attention may beturned to the environment around the vehicle, thereby ensuring a safedriving for parking.

The vehicle parking assist device further comprises display guide meansfor visually presenting the guide by the parking assist together withthe image by the camera.

In the vehicle parking assist device, the visual presentation of theguide by the parking assist is performed together with the image by thecamera. Therefore, the driver moves the vehicle for parking withoutanxiety while watching the image of the dead space when viewed from thedriver.

According to the invention, there is provided a vehicle parking assistdevice for assisting a driver when he backs a vehicle into a parkingspace, by use of image information, the vehicle parking assist devicecomprises:

a camera for picking up a scene of a backward area of the vehicle;

display means for displaying the image of the backward area picked up bythe camera; and

parking assist means for assisting the driver in his parking by use of adisplay on the screen of the display means in which the guide linesprolonged from the lines defining the width of the vehicle and distancelines are overlaid on the backward area image.

When the camera photographs a backward area of the vehicle, an image ofthe backward area is displayed on the screen of the display means. Theparking assist means displays the guide lines prolonged from the linesdefining the width of the vehicle and distance lines in a state thatthose lines are overlaid on the backward area image. Since the guidelines are prolonged from of the lines defining the width of the selfvehicle, the driver can approximately grasp a distance of the backingself vehicle to the object with reference to the distance lines afterthe vehicle is directed to be parallel to the boundary lines of theparking space in which the self vehicle is to be parked and otherparking spaces adjacent to the former, and easily grasps a distance thatthe vehicle runs till the parking is ended and the vehicle is stopped.

According to the invention, there is provided a vehicle parking assistdevice for assisting a driver when he backs a vehicle into a parkingspace, by use of image information, the vehicle parking assist devicecomprises:

a camera for picking up a scene of a backward area of the vehicle;

display means for displaying the image of the backward area picked up bythe camera;

a steering angle sensor for detecting a steering angle of the selfvehicle;

traveling path predicting means for predicting a traveling path of theself vehicle on the basis of a steering angle detected by the steeringangle sensor and for calculating a predictive traveling path curverepresenting the result of the prediction;

parking position judging means for judging a parking position at whichthe self vehicle is to be parked; and

parking assist means for effecting a parking assist in such a way that alength of a predictive traveling path curve calculated by the travelingpath predicting means is adjusted in accordance with a distance rangingto a parking position judged by the parking position judging means, andthe predictive traveling path curve length adjusted is overlaid on thebackward area image on the screen of the display means.

In this vehicle parking assist device, a length of a predictivetraveling path curve that is predicted on the basis of a steering angleof the self vehicle is adjusted in accordance with a distance ranging toa parking position at which the self vehicle is to be parked. Therefore,the end of the predictive traveling path curve is coincident with theparking position, and hence the parking assist is performed by using adisplay which is easy for the driver to watch.

According to the present invention, there is provided a vehicle parkingassist device for assisting a driver when he backs a vehicle into aparking space, by use of image information, the vehicle parking assistdevice comprises:

a camera for picking up a scene of a backward area of the vehicle;

display means for displaying the image of the backward area picked up bythe camera;

a steering angle sensor for detecting a steering angle of the selfvehicle;

traveling path predicting means for predicting a traveling path of theroof portion of the self vehicle on the basis of a steering angledetected by the steering angle sensor and for calculating a predictivetraveling path curve representing the result of the prediction; and

parking assist means for effecting a parking assist in such a way that apredictive traveling path curve of the roof portion of the vehicle,which is calculated by the traveling path predicting means, is overlaidon the backward area image on the screen of the display means.

In this vehicle parking assist device, a roof predictive traveling pathcurve of the roof portion of the self vehicle is displayed whileoverlaid on the backward area image for the parking assist. Therefore,the vehicle can be backed while being unconscious of the side of thevehicle adjacent to the parking space which forms the dead space whenthe driver backs the vehicle along the predictive traveling path curveso as not to touch the roof of the vehicle being parked in the parkingspace to which the parking space is adjacent.

According to the present invention, there is provided a vehicle parkingassist device for assisting a driver when he backs a vehicle into aparking space, by use of image information, the vehicle parking assistdevice comprises:

a camera for picking up a scene of a backward area of the vehicle;display means for displaying the image of the backward area picked up bythe camera;

a steering angle sensor for detecting a steering angle of the selfvehicle;

traveling path predicting means for predicting a traveling path of theself vehicle on the basis of a steering angle detected by the steeringangle sensor and for calculating a predictive traveling path curverepresenting the result of the prediction;

vehicle projection means arranged such that the vehicle projection meansjudges if other vehicles park in parking spaces located adjacent to aparking space under photographing by the camera, and when other vehiclespark therein, the vehicle projection means calculates a projective lineof the vehicle on the ground; and

parking assist means for effecting a parking assist in such a way that apredictive traveling path curve calculated by the traveling pathpredicting means and a projective line of the vehicle to the groundcalculated by the vehicle projection means are overlaid on the backwardarea image on the screen of the display means.

In this vehicle parking assist device, in case where a scene of aparking space into which the self vehicle is backed is picked up by thecamera, when another vehicle is parked in a parking space adjacentthereto, the projective line of the vehicle to the ground is calculated,and the calculated projective line and a predictive traveling path curveare overlaid on the backward area on the screen of the display means,and in this state, the parking assist is performed. Therefore, reliablejudgement is made as to if there is an area which hides behind anothervehicle, being dead space of the image, affects the predictive travelingpath curve of the other vehicle. Therefore, when the self vehicle isguided to the parking area, reliable judgement is made as to if thevehicle may move into the parking area without touching the othervehicle even in a shaded area in the image of the other vehicle.

According to the present invention, there is provided a vehicle parkingassist device for assisting a driver when he backs a vehicle into aparking space, by use of image information, the vehicle parking assistdevice comprises:

a camera for picking up a scene of a backward area of the vehicle;

display means for displaying the image of the backward area picked up bythe camera;

plane model generating means for generating from the backward area imagepicked up by the camera a plane model containing a parking space inwhich a self vehicle is to be parked and the parking spaces locatedadjacent to the parking space, together with the self vehicle and othervehicles being parked; and

parking assist means for effecting a parking assist by displaying aplane model generated by the plane model generating means on the screenof the display means.

In this vehicle parking assist device, the parking space in which theself vehicle is to be parked, and the parking spaces adjacent to theformer are modeled in a plane, and those are displayed in the form of aplane model. Further, the parking spaces of the self vehicle and othervehicles, which are parked in those parking areas, are also displayed inthe plane model. Therefore, the parking spaces and the vehicles may bedisplayed in a bird's eye view, and this provides an easy-to-understandparking assist. Since the parking spaces and the vehicles are displayedin a plane, the driver can easily grasp positions of the self vehiclerelative to the parking areas and the vehicles parked adjacent to theformer, and the orientation of the vehicle and its advancing direction.

In the vehicle parking assist device, the parking assist means displaysan image on the screen of the display means so that the backwarddirection of the self vehicle is oriented downward.

Thus, the backward direction of the self vehicle is thus orienteddownward. This display is well fit to the feeling of the driver.Therefore, when the driver watches the screen of the display means, heis easy to understand the contents displayed on the screen.

In this vehicle parking assist device, the parking assist means iscapable of selectively displaying the backward area image output fromthe camera and the parking assist image or the plane model to bedisplayed overlaid on the backward area image on the screen of thedisplay means

In this vehicle parking assist device, the parking guide may be switchedbetween the parking guide modeled in place or the parking guide basedthe backward area image. Therefore, the driver selects, as desired, theparking guide modeled in place or the actual parking guide based thebackward area image, and it effects the parking assist which is easy forthe driver to understand.

According to the present invention, there is provided a vehicle parkingassist device for assisting a driver using image information when thedriver backs a vehicle into a parking space, the vehicle parking assistdevice comprises:

a camera for picking up a scene of a backward area of the vehicle;

display means for displaying the image of the backward area picked up bythe camera;

a timer for counting a predetermined time;

direction detecting means for detecting a change of the advancingdirection of the vehicle; and

parking assist means arranged such that when the direction detectingmeans detects a change of the advancing direction of the vehicle to thebackward direction, the parking assist means displays the backward areaimage picked up by the camera on the screen of the display means toassist the driver in his parking operation, and when the directiondetecting means detects a change of the advancing direction of thevehicle from the backward direction to the forward direction, theparking assist causes the timer to start the counting of time, andcontinues the parking assist based on the backward area display on thescreen of the display means till a time set in the timer elapses.

In this vehicle parking assist device, when the vehicle backs, theparking assist based on the backward area image is carried out. Even ifthe advancing direction of the vehicle is changed from the backwarddirection to the forward direction, the parking assisting operationcontinues with the display of the backward area image till the time setin the timer terminates. Therefore, the backward area is easily checkedwhen the vehicle advances and when the steering wheel is reverselyturned. Further, when the steering wheel is reversely turned, thevehicle parking assist device displays a yard stick in estimating, bythe eye, what distance the vehicle must be advanced in aneasy-to-understand manner, for the parking assist.

According to the invention, there is provided a vehicle parking assistdevice in which when a driver backs a vehicle into a parking spacebetween adjacent vehicles of those being parked in a row, the vehicleparking assist device assists the driver in his parking operation byimage-information basis guide, the vehicle parking assist devicecomprising:

a camera for picking up a scene of a backward area of the vehicle;

a steering angle sensor for detecting a steering angle to steer thevehicle;

traveling path predicting means for predicting a traveling path of thevehicle on the basis of a steering angle detected by the steering anglesensor and for calculating a predictive traveling path curve;

timing judging means for detecting a target parking position on thebasis of the image picked up by the camera and for judging a timing toreversely turn a steering wheel by an amount of turn of the steeringwheel necessary to move the vehicle to the target parking position;

parking assist means for generating a composite image of an imagederived from the camera and a predictive traveling path curve predictedby the traveling path predicting means, and for adding guide informationfor a steering operation defined by a reverse-turn timing judged by thetiming judging means to the composite image, whereby the parking assistmeans assists the driver in the serial parking operation; and

display means for display an image derived from the parking assistmeans.

The scene of a backward area of the vehicle is picked by a camera, andthe steering angle to steer the vehicle is detected by a steering anglesensor. The traveling path predicting means predicts a traveling path ofthe vehicle on the basis of a steering angle detected by the steeringangle sensor and calculates a predictive traveling path curve. Thetiming judging means detects a target parking position on the basis ofthe image picked up by the camera and judges a timing to reversely turna steering wheel by an amount of turn of the steering wheel necessary tomove the vehicle to the target parking position. The parking assistmeans generates a composite image of an image derived from the cameraand a predictive traveling path curve predicted by the traveling pathpredicting means, and adds guide information for a steering operationdefined by a reverse-turn timing judged by the timing judging means tothe composite image, whereby the parking assist means assists the driverin the serial parking operation. A predictive traveling path curve isdisplayed together with a backward area image of the vehicle. Further,guide information showing such a timing of reversely turning thesteering wheel as to back the vehicle into a target parking position isalso displayed. Accordingly, the driver easily drives the vehicle forthe series parking while watching the screen of the information displaydevice.

In the invention, the parking assist means adds an oblique line as theguide information for the steering operation to a position on apredicted traveling path curve at which the steering wheel is to bereversely turned.

Thus, the parking assist means adds an oblique line as the guideinformation for the steering operation to a position on a predictedtraveling path curve at which the steering wheel is to be reverselyturned. Accordingly, the parking is completed by reversely turning thesteering wheel when the oblique line lies on a target, e.g., a roadedge. Therefore, the driver readily makes a series parking withouttouching of the vehicle with another vehicle being parked adjacent tothe vehicle driven by the driver.

The parking assist means assists the driver in the series parking on thebasis a backward movement of the vehicle and a combination of lightingconditions of a hazard lamp and/or winker lamp.

In the invention, the parking assist means assists the driver in theseries parking on the basis a backward movement of the vehicle and acombination of lighting conditions of a hazard lamp and/or winker lamp.Therefore, there is no need of switching operation for starting theseries parking assist. The series parking assist is automaticallycarried out if the hazard lamp and/or winker lamp is lit to give thedrivers of other vehicles a sign showing that the series parking willstart. In this respect, there is eliminated troublesome switchingoperations.

In the invention, the parking assist means judges which side of the roadis used for the series parking on the basis of a combination of lightingconditions of the hazard lamp and/or winker lamp, and adds the guideinformation to the composite image according to the judgement result.

Thus, the parking assist means judges which side of the road is used forthe series parking on the basis of a combination of lighting conditionsof the hazard lamp and/or winker lamp, and adds the guide information tothe composite image according to the judgement result. When one of thewinker lamps is lit, it is judged that the vehicle will be series parkedalong the same side of the road as the side of the car to which the litwinker lamp is attached, and the series parking guide informationprepared for such a series parking is attached to the composite image.When the hazard lamp is lit, it is judge that the vehicle will be seriesparked on the left side of the road as usually, and the vehicle parkingassist device assists the driver to drive the vehicle for parking. Sincethe parking assist means assists the driver in making the seriesparking, the parking assist of the device is free from the complicatedswitching operations.

The parking assist means generates an ideal traveling path ranging fromthe present position to a target parking position detected by the timingjudging means, for the steering guide information.

Thus, an ideal traveling path ranging from the present position to atarget parking position detected by the timing judging means, isgenerated for the steering guide information. Therefore, the drivereasily backs the vehicle into a target parking position by steering thevehicle so as to approximate in shape the predictive traveling pathcurve to the ideal traveling path.

The vehicle parking assist device further comprises lane monitoringmeans for recognizing border lines of a traffic lane and monitoring anout-of-lane of the vehicle on the basis of the border line recognition.

In the vehicle parking assist device, lane monitoring means recognizesborder lines of a traffic lane and monitors an out-of-lane of thevehicle on the basis of the border line recognition. Therefore, onecamera may be used for both the camera which monitors a backward area ofthe vehicle for parking assist and the camera for monitoring anout-of-lane of the vehicle. Therefore, the vehicle parking assist deviceneeds less cost when comparing with the vehicle parking assist deviceusing two cameras for the above purposes.

In the vehicle parking assist device, the parking assist means judgeswhether or not the vehicle may back to a target parking positiondetected by the timing judging means within a range of steering angleswithin which the steering wheel may be turned, and whether or not thevehicle moves out of the traffic lane monitored by the lane monitoringmeans, and the parking assist means assists the driver in the seriesparking on the basis of the judgement results.

In the invention, the vehicle may back to a target parking positionwithin a range of steering angles within which the steering wheel may beturned. Further, judgement is made as to whether or not the vehiclemoves out of the traffic lane monitored by the lane monitoring means.The parking assist means assists the driver in the series parking on thebasis of the judgement results. Therefore, the driver may back thevehicle into a parking space between the adjacent vehicles of thosebeing parked in row, without getting the vehicle out of the lane.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view schematically showing a vehicle parkingassist device which is a first embodiment of the invention.

FIG. 2 is a perspective view schematically showing a state that avehicle 1 backs for parking in the FIG. 1 embodiment.

FIG. 3 is a block diagram showing a parking assist ECU 6 in the FIG. 1embodiment.

FIGS. 4A to 4C are diagrams showing parking assist displays displayed onthe screen of an information display device 4.

FIG. 5 is a diagram showing a navigation display 41 and a audio display42, which are presented on the screen of the information display device4.

FIG. 6 is a diagram showing a display by the information display device4, which informs that a shift lever is shifted to the reverse gearposition.

FIG. 7 is a diagram showing a display by the information display device4, which informs that an obstructive object is detected, the parkingoperation is completed, and urges the driver to make a check by the eye.

FIG. 8 is a block diagram showing a scheme of a vehicle parking assistdevice which is a second embodiment of the invention.

FIG. 9 is a diagram showing a parking assist display on the screen ofthe information display device 4 in the FIG. 8 embodiment.

FIG. 10 is a block diagram showing a scheme of a vehicle parking assistdevice which is a second embodiment.

FIG. 11 is a plan view showing a plurality of camera units 10 disposedaround a body of a vehicle 1, which is a fourth embodiment of theinvention.

FIGS. 12(1) to 12(12) are diagrams exemplarily each showing segmenteddisplay areas on the screen of the information display device 4 in theFIG. 11 embodiment, images output from the camera units 10 beingsimultaneously displayed in those segmental areas.

FIG. 13 is a diagram exemplarily showing segmental display areas on thedisplay screen and a display containing a vehicle illustration 70 in theFIG. 11 embodiment.

FIG. 14 is a diagram showing a state that a position of the vehicleillustration 70 is shifted in accordance with a camera image in the FIG.11 embodiment.

FIG. 15 is a diagram showing a vehicle illustration 70 and the directionand area of a camera image in the FIG. 11 embodiment.

FIG. 16 is a diagram showing a camera image 71 being divided into anavigation display 41 and an audio display 42 in the FIG. 11 embodiment.

FIG. 17 is a diagram showing a state in which a plurality of imagedisplays are changed one to another by a wipe method in the FIG. 11embodiment.

FIG. 18 is a diagram showing a segmented screen containing images inwhich segmental display areas and display positions are varied inaccordance with a steering angle of a steering wheel.

FIG. 19 is a block diagram showing a scheme of a vehicle parking assistdevice which is a fifth embodiment of the invention.

FIGS. 20A, 20B and 20C are diagrams showing how obstacle displays arechanged to another on the screen of an information display deviceinformation display device 74.

FIGS. 21A and 21B are diagrams showing a drive assist which is carriedout by seamlessly composing the images picked up by a plurality ofcamera units 10 into a single image, the drive assist forming a sixthembodiment of the invention.

FIGS. 22A and 22B are diagrams showing how to display images, which areproduced by picking up the entire circumferential area around a vehicle1, on the display screen in the FIGS. 21A and 21B embodiment.

FIG. 23 is a block diagram showing a scheme of a vehicle parking assistdevice which is a seventh embodiment of the invention.

FIGS. 24A and 24B are diagrams showing displays on the screen of ainformation display device 4, 74 in which a brightness of the display inthe forward movement of the vehicle is different from that of thedisplay in the backward movement, in each of the embodiments.

FIG. 25 is a diagram showing a backward area display presented within adisplay frame 87, which clearly shows the driver that the image beingdisplayed is that of the backward area.

FIG. 26 is a block diagram showing a basic arrangement to carry out aparking assist in eighth to 11th embodiments of the present invention.

FIG. 27 is a diagram showing a state that a vehicle 101 backs to parkingzone 102 and parks there.

FIG. 28 is a block diagram showing an electrical arrangement of aparking assist ECU 106 in FIG. 26.

FIG. 29 is a diagram showing, in a simplified form, an image to bedisplayed by an information display device 104 in the parking assist ofthe eighth embodiment of the invention.

FIG. 30 is a flow chart showing a control procedure in the FIG. 29embodiment.

FIG. 31 is a diagram showing, in a simplified form, an image to bedisplayed by an information display device 104 in the parking assist ofthe ninth embodiment of the invention.

FIG. 32 is a flow chart showing a control procedure in the FIG. 31embodiment.

FIG. 33 is a diagram showing, in a simplified form, an image to bedisplayed by an information display device 104 in the parking assist ofthe tenth embodiment of the invention.

FIG. 34 is a flow chart showing a control procedure in the FIG. 33embodiment.

FIG. 35 is a diagram showing, in a simplified form, an image to bedisplayed by an information display device 104 in the parking assist ofthe eleventh embodiment of the invention.

FIG. 36 is a flow chart showing a control procedure in the FIG. 35embodiment.

FIG. 37 is a block diagram schematically showing an arrangement of avehicle drive assist device according to a 12th embodiment of thepresent invention.

FIG. 38 is a diagram showing a state that by the utilization of thevehicle drive assist device of FIG. 37, the driver backs the vehicle 201to a parking space marked off with white lines 203 and parks it there.

FIG. 39 is an electrical arrangement of the parking assist ECU 206 shownin FIG. 37.

FIG. 40 is a flow chart showing a parking assist procedure, which iscarried out by the parking assist ECU 206 in the 12th embodiment.

FIG. 41 is a block diagram showing a basic arrangement for a driveassist in each embodiment of the present invention.

FIG. 42 is a diagram showing how a vehicle 301 of FIG. 41 backs to aparking zone 302 and parks therein.

FIG. 43 is a block diagram showing an electrical arrangement of aparking assist ECU 306 in FIG. 41.

FIG. 44 is a diagram roughly showing an image displayed on the screen ofan information display device 304 in a parking assist of the firstembodiment of the invention.

FIG. 45 is a diagram roughly showing an image displayed on the screen ofan information display device 304 in a parking assist of the secondembodiment of the invention.

FIG. 46 is a diagram roughly showing an image displayed on the screen ofan information display device 304 in a parking assist of the thirdembodiment of the invention.

FIG. 47 is a diagram roughly showing an image displayed on the screen ofan information display device 304 in a parking assist of the fourthembodiment of the invention.

FIG. 48 is a diagram roughly showing an image displayed on the screen ofan information display device 304 in a parking assist of the fifthembodiment of the invention.

FIG. 49 is a block diagram showing an electrical arrangement of aparking assist ECU 366 used in a sixth embodiment of the invention.

FIG. 50 is a flow chart showing a control procedural in the FIG. 50embodiment.

FIG. 51 shows an external appearance of a vehicle which will undergo aseries parking assist in 19th and 20th embodiments of the presentinvention.

FIG. 52 is a plan view showing a basic arrangement for a drive assistwhen a series parking of the vehicle of FIG. 51 starts.

FIG. 53 is a block diagram showing an electrical arrangement of aparking assist ECU 406 shown in FIG. 51.

FIG. 54 schematically shows an image display for the series parkingassist, which is carried out as the 19th embodiment of the presentinvention.

FIG. 55 shows a control procedure in the embodiment of FIG. 54.

FIG. 56 is a diagram showing a series parking assist image in 20thembodiment of the present invention.

FIG. 57 is a flow chart showing a control procedure by the DSP in theFIG. 56 embodiment.

DETAILED DESCRIPTION OF THE PRESENT INVENTION

FIGS. 1 and 2 are diagrams showing a scheme of a vehicle parking assistdevice which is a first embodiment of the present invention. Likereference numerals are used, for simplicity, for designating like orequivalent portions through the drawings which will be referred to inthe description of the preferred embodiments of the invention.

The vehicle parking assist device of the embodiment assists the driverto correctly park a vehicle into a parking space marked off with whitelines 3 or the like, through the backward movement of the vehicle. Adrive assist displays a predictive traveling path curve 5 a representingas a predictive traveling path on the screen of an information displaydevice 4 as display means, or white lines 5 b prolonged from the linesdemarcating both sides of a vehicle 1 on the display screen, whereby adrive assist for the driver is performed.

The predictive traveling path curve 5 a and the white lines 5 b aregenerated by an electronic control unit (referred to as a “parkingassist ECU 6”) for the parking assist by use of detection data of asteering angle and the like. The generated ones are displayed by theinformation display device 4.

When the driver operates a shift lever 8 of a transmission of thevehicle 1 to change the advancing direction of the vehicle to a backwarddirection, the parking assist ECU 6 causes the information displaydevice 4 to display a predictive traveling path curve 5 a and whitelines 5 b on the screen thereof. At this time, a speaker 9 produces analarm sound or a voice sound for guide.

An image picked by a camera unit 10 attached to the rear part of thevehicle 1 is also displayed on the screen of the information displaydevice 4, in addition to the predictive traveling path curve 5 a and thewhite lines 5 b generated by the parking assist ECU 6. In the imagepicked up through the field 10 a of the camera unit 10, an areal portionwhich is not clearly viewed by the driver or is a dead space as viewedfrom the driver is clearly viewed. The image assists the driver inparking the vehicle 1 into the parking zone 2 in an easy-to-understandmanner, in cooperation with the predictive traveling path curve 5 a andthe white lines 5 b.

A direction of the predictive traveling path curve 5 a generated by theparking assist ECU 6 is varied in accordance with a steering operationof the steering wheel 7. A steering operation of the steering wheel 7 isobtained by detecting an angular displacement of a steering shaft 11 byan angle sensor 12. In a normal vehicle 1, the steering wheel 7 may beturned by several turns. Therefore, the angle sensor 12 for detecting asteering angle detects an angular displacement and its direction of thesteering shaft 11, detects a state of the steering shaft 11 positionedat the center position.

In the display, the white lines 5 b are overlaid on the image picked upby the camera unit 10. In this case, the white lines 5 b are prolongedfrom the lines defining the body width of the vehicle 1. The directionof the white lines 5 b is fixed with respect to the field 10 a of thecamera unit 10. The vehicle 1 is equipped with a con-light sensor 14which senses a brightness of the environment and controls a lightingcondition of an illumination system. A brightness and coloring of thepredictive traveling path curve 5 a and the white lines 5 b, which aredisplayed by the information display device 4, are varied in accordancewith a brightness of the environment of the vehicle 1. Thus, those areadjusted so as to provide an easy-to-understand drive assist.

The vehicle 1 is also equipped with car-carried devices, such as anavigation device 15 and an audio device 16, and necessary informationis displayed by the information display device 4. The vehicle 1 isfurther equipped with a back sonar 17 and a corner sensor 18 as obstaclesensors. An obstructive object existing in the dead space by theutilization of ultrasonic waves.

FIG. 3 shows the innards of the parking assist ECU 6 shown in FIGS. 1and 2. The parking assist ECU 6 contains a digital signal processor(DSP) for controlling the overall system of the device. The DSP 20carries out controls and signal processings through a bus 21. A signalrepresenting an image picked up by the camera unit 10, which is of theNTSC type, is input to an amplifier/filter circuit 22. Then, it is inputto an analog to digital converter (ADC) 23. The ADC 23 converts a signalform of the image signal into a digital form. The digitized signal isstored into an amplifier/filter circuit 24. A video signal output fromthe amplifier/filter circuit 24 is also applied to a sync separationcircuit 25 where a horizontal sync signal and a vertical sync signal areseparated from the video signal. Those separated ones are input the DSP20.

The DSP 20 receives an angle detect signal from the angle sensor 12 fordetecting an angular displacement of the steering shaft 11 and a centerposition detect signal from the center position sensor 13. The DSP 20carries out a parking assist process by use of programs and data storedin a program memory 27 and a data memory 28, both being connected to thebus 21. The storage memory 29, which is backed up by a battery, iscapable of retaining the contents stored therein even when a powersupply 35 of the parking assist ECU 6 is not turned on.

The DSP 20 recognizes the white lines 3 on the basis of an incomingvideo signal, and generates a predictive traveling path curve 5 a whenan angular displacement operation of the steering shaft 11 is performed,and white lines 5 b. The generated images are stored into field buffers31 and 32, either of which is selected by a switch (SW) circuit 30. Thesignal selected by the SW circuit 30 is input to a digital to analogconverter (DAC) 33, and passes through a filter/amplifier circuit 34,and is output as a video signal to the information display device 4.Electric power is supplied to the whole system of the parking assist ECU6. When a reset signal is supplied to the parking assist ECU by thereset circuit 36 when the power supply is turned on. Further, a clocksignal defining a basic operation timing of the parking assist ECU andsignals having frequencies formed by demultiplying the frequency of theclock signal are also applied to the parking assist ECU.

Each of FIGS. 4A to 4C exemplarily shows image information for parkingassist, which is displayed on the screen of the information displaydevice 4. FIG. 4A shows an image corresponding to a state that thevehicle 1 is backed into a parking position 40. The white lines 5 bprolonged from the lines defining the vehicle width are displayed on thebackward image, together with a predictive traveling path curve 5 apredicted on the basis of a steering angle. As shown in FIG. 4B, onlythe predictive traveling path curve 5 a is added to the backward imagefor the parking position 40, or only the backward image is added theretoas shown in FIG. 4C. Further, only the white lines 5 b may be overlaidon the backward image shown in FIG. 4C.

The driver can know the present direction of the vehicle 1 under parkingassist from a display of the white lines 5 b. Where the white lines 3are drawn on both sides of the parking position 40 to form a parkingframe, the driver can easily park the vehicle 1 in parallel with a spaceof the parking position 40 in a manner that when the white lines 3 areparallel to the white lines 5 b, the driver returns the steering wheel 7to the center position.

As shown in FIGS. 4A to 4C, a display of the predictive traveling pathcurve 5 a or the white lines 5 b may be selected as desired by a selectswitch provided for selecting a display mode. In a specific example, thepredictive traveling path curve 5 a is displayed for a fixed time onlywhen a predetermined driving operation, e.g., an operation of thesteering wheel 7, is performed. The white lines 5 b is displayed for afixed time only when the braking operation is performed. Thoseoperations are automatically controlled.

A brightness and color arrangement of the predictive traveling pathcurve 5 a and the white lines 5 b are varied in accordance with abrightness of the environment of the vehicle 1, which is sensed by thecon-light sensor 14. For example, when the environment is dark, thedisplay is also dark, whereby the backward image is in harmony with thepredictive traveling path curve 5 a and the white lines 5 b. The colorsof the predictive traveling path curve 5 a and the white lines 5 b arealso adjusted in accordance with a brightness and the coloring of theenvironment so that the driver is easy to see them. Since in thedaytime, the environment scene is bright, a distinctive color, e.g.,yellow, is used. At night, the environment is dark. Therefore, if adistinctive color is used, the backward image is difficult to see.Accordingly, in this case, a dark and thin color is used.

In the embodiment, the con-light sensor 14 is used for inputting abrightness of the environment to vary the brightness and the color ofthe predictive traveling path curve 5 a and the white lines 5 b. Ifrequired, a sensor exclusively used for the brightness sensing purposemay be used. The brightness may be detected from an image of a picturepicked up by the camera unit 10. When the camera unit 10 is capable ofpicking up a color image, a color is also detected from the image. Inthis case, the colors of the predictive traveling path curve 5 a and thewhite lines 5 b may be changed while being interconnected in operationwith the detected color.

In a case where as shown in FIGS. 4A to 4C, the parking assist image ischanged to another by the operation of the switch by the driver, thefinal select state in the previous parking assist operation is storedinto the storage memory 29 and the first select state in the parkingassist operation now conducted is set on the basis of the final selectstate stored. If the select state providing an easy-to-see display forthe driver is obtained through the above operation, that select statemay be used for the next parking assist and the subsequent ones.

FIG. 5 shows a display by the information display device 4, the displaycontaining a navigation display 41 from the navigation device 15 in FIG.1 and an audio display 42 from the audio device 16. The driver mayselect, as needed, the backward image of the parking position 40 forparking assist, the navigation display 41 or audio display 42, wherebythe information display device 4 is effectively used for display.

FIG. 6 shows a display by the information display device 4, whichinforms the driver that the shift lever 8 is shifted to the reverse gearposition. A telop 43 of “Reverse Gear” is displayed for a fixed time onthe backward image representing the parking position 40. The driver seesthe telop 43, and easily understands that the shift lever is at thereverse gear position. With the telop displayed, there is no chance thatthe shift position slips off the driver's memory and the drivermistakenly recognizes that the vehicle is in the forwarding state, andabruptly starts the forward movement of the vehicle. The color of theentire screen may be changed in place of the display of the telop ortogether with the display of the telop 43.

FIG. 7 shows a display containing a telop 44 showing an obstacledetection by the back sonar 17 or corner sensor 18, a telop 45 showingthe parking end, and a telop 46 urging the driver to make a check by theeye. When the back sonar 17 or the corner sensor 18 detects anobstructive object located near the vehicle 1, a telop 43 of “obstacle”or “STOP”, showing that an obstacle is present for a predetermined time,and urges the driver to turn his attention to the obstructive object.The obstructive object detected by the back sonar 17 or the cornersensor 18 could be contained in the backward image near the parkingposition 40. In this case, the driver sees the telop 44 and watches thebackward image. If so done, he will readily recognize the obstructiveobject 47 by the eye. When the obstructive object 47 is detected, thecolor of the whole screen may be changed in place of the display of thetelop 44 or in combination of the display of the telop 44, to call thedriver's attention to the obstructive object.

The telop 45 of the parking end is displayed when the obstacle sensor,such as the back sonar 17 or the corner sensor 18, detects a car stopperor a wall, and the vehicle approaches to the detected one within apredetermined distance from the detected one. Display of the telop 45 of“parking end” or “STOP” informs the driver of the parking end, therebypreventing a vehicle touch accident in the backward movement. The telop46 causing the driver to make a check by the eye is displayed for urgingthe driver to make a safety check by the eye during the execution of theparking assist process. Display of the telop 46 prevents such adangerous situation that the driver is riveted to the display by theinformation display device 4 and gives less attention to the environmentincluding the frontward and sideward areas of the vehicle 1, which isnot contained in the image displayed by the information display device4.

Improvement of the safety during the backward movement of the vehiclemay be achieved in a manner that when the shift lever is shifted to thereverse gear position, a vehicle speed is controlled to be below apredetermined speed. The speed control may be performed also when theobstacle sensor, e.g., the back sonar 17, detects an obstructive object47. The control to limit the vehicle speed to below a predeterminedspeed during the backward movement or the backward movement when theobstructive object is detected is capable of preventing a vehiclecollision during the backward movement and lessening an impact at thetime of the collision.

FIG. 8 shows a scheme of a vehicle parking assist device which is asecond embodiment of the invention. In the figure, like or equivalentportions are designated by like reference numerals in the description ofthe first embodiment. A camera unit 50 used in the second embodiment isdesigned such that its field 50 a is varied by use of a remotecontroller 51. The camera unit 50 is attached to the rear part of thevehicle 1. A back light 52 for illuminating an area covered by the field50 a of the camera unit 50 at night is also used. The back light 52 isdirected to the field 50 a of the camera unit 50 in accordance with aremote control of the camera unit 50 by the remote controller 51. In theembodiment, a smooth parking may be secured by executing a throttlecontrol 53 and a brake control 54 by the parking assist ECU 6, inaddition to the parking assist by use of the information display device4. Parking assist information using a predictive traveling path curve 55a and guide lines 55 b is displayed on the screen of the informationdisplay device 4.

A back light mounted on the vehicle 1 may be used for the back light 52.It is preferable that the back light 52 is optionally contained in theaccessories of the camera unit 50. In particular when a mechanism tochange the illumination direction is provided, optional use of it iseasy. Where the illumination direction can be changed, the followingadvantage is obtained: when the shift lever 8 of the transmission isshifted to the reverse gear position, the back light 52 is turneddownward to improve the visibility in a dark place, e.g., at night.

FIG. 9 shows an image for a parking assist displayed in the FIG. 8embodiment. In the embodiment, the guide lines 55 b are curved inaccordance with an image distortion by a wide angle camera. For a cameraof the camera unit 50, a wide angle lens is used for securing a field 50a as wide as possible. In the backward image of the parking position 40or the like, an actual straight portion is picked up as a curved image.Accordingly, the curving of the guide lines 55 b in accordance with adistortion of the image of the guide lines 55 b will ensure a more exactparking assist.

When a part, e.g., bumper, of the vehicle body of the vehicle 1 iscontained in the field 50 a of the camera unit 50, the vehicle body ismore closer to this side, so that the body width of the vehicle 1 islarger than the distance between the guide lines 55 b since the image isdistorted. As for the predictive traveling path curve 55 a, if it isdisplayed having the width equal to the vehicle width, when a part ofthe vehicle body is seen, the body width of the vehicle appears to belarger than the guide line distance in a region near the vehicle body,as in the case of the guide lines 55 b. In the embodiment, when thepredictive traveling path curve 55 a and the guide lines 55 b aredisplayed corresponding to the body width of the vehicle 1, thenon-coincident portion near the body of the vehicle 1 is not displayed,and the driving for the non-coincident portion depends on the judgementof the driver.

For the non-coincident portion near the vehicle body, the predictivetraveling path curve 55 a and the guide lines 55 b may be increased inwidth to be continuous to both sides of the vehicle body when those aredisplayed. Those widely displayed portions of the predictive travelingpath curve 55 a and the guide lines 55 b are wider than the actualvehicle body. Accordingly, if another vehicle and an obstructive objectare not present in those widened portions, a more reliable safety issecured.

As described, above, the vehicle 1 may be parked in the camera unit 50in a state that the steering wheel is positioned at the center positionwhen the guide lines 55 b are parallel to the sides of a vehicle beingparked at another parking position or while lines drawn in the parkingposition 40.

If the width of the guide lines 55 b is wider than the body width of thevehicle 1, a parking assist for parking the vehicle 1 into the parkingposition 40 is easier to understand. For example, the width of the guidelines 55 b is wider than the actual vehicle width by a fixed distance.The vehicle may be parked in a state that the steering wheel is set in astraight drive state in a manner that when the guide lines 55 b lie onthe white lines in the parking position 40, boundary lines between itand the adjacent parking positions, or the sides of the vehicles beingalready parked in the adjacent parking positions, the steering wheel isreturned to the center position. Further, the width of the predictivetraveling path curve 55 on the display may be selected in accordancewith the conditions of the parking zone, which is daily used by thedriver. In the case of the daily used parking zone, a certain fixeddistance may be a distance value suitable for the parking position 40.When the parking zone is not fixed in use, the certain fixed distance isa minimum value necessary for the opening/closing of the door, and thegoing in and out, so as to secure a space allowing the driver to go outof the vehicle 1.

In FIG. 9, a line indicating a stop position 57 of the vehicle 1 isdisplayed on the guide lines 55 b near the vehicle 1. In the backwardimage, a line is placed at a position apart from the rear end of thevehicle 1 by a fixed distance. And when the line or the like of theparking frame of the parking position 40 reaches that position, thevehicle is stopped. The fixed distance is preferably selected to havesuch a distance as to avoid the touching of that the bumper or the likeof the rear end of the vehicle 1 with an obstructive object, the carstopper, or the like and to be as short as possible, in order that thevehicle may be parked in a narrow parking space.

Dotted lines are used to depict the guide lines 55 b on the display soas to be less noticeable than the predictive traveling path curve 55 a.In other words, the predictive traveling path curve 55 a is intensifiedin brightness. As a result, an easy-to-understand drive assist isprovided to the driver. The length of the guide lines 55 b may be variedwhile being interconnected in operation with the back sonar 17. In aspecific example, the length of the guide lines 55 b is reducedaccording to a level of an alarm issued when the back sonar 17 detectsan obstructive object 47. As a result, the driver gains a reference inmeasuring the vehicle moving distance, and may have a drive assist ofactual drive feeling.

FIG. 10 shows a scheme of a vehicle parking assist device which is athird embodiment of the invention. In the embodiment, a plan model 60 isprovided to select a series parking mode for a series parking assist 61or a normal parking mode for a normal parking assist 62 on the screen ofthe information display device 4. In the normal parking mode, the field50 a of the camera unit 50 and the illumination by the back light 52 aredirected mainly to the backward area of the vehicle 1. In the seriesparking mode, the field 50 a of the camera unit 50 and the illuminationby the back light 52 are directed to the side of the vehicle.

The series parking mode or the normal parking mode may be automaticallyselected. The series parking mode may be selected when the turning on ofthe winker lamp 63 or the hazard lamp 64 and the shifting of the shiftlever 8 to the reverse gear position are simultaneously performed. In analternative, the driver refers to map data about the present position ofthe vehicle 1, which is presented by the navigation device 15. When thepresent position of the vehicle is in the parking zone, the normalparking mode is selected. When it is on a road, the series parking modeis selected. In another alternative, a change of a steering angle of thesteering wheel 7 is used to judge as to whether or not the parking modeis the series parking mode. In the series parking, it is frequent thatthe vehicle 1 is steered by a special steering operation, such as areverse turn of the steering wheel. Typical steering patterns are storedinto the storage memory 29. When the present steering pattern iscoincident with any of the stored ones, the process judges that theseries parking will be performed, and the process is designed such thatthe parking mode is automatically changed to the series parking mode.

FIG. 11 is a diagram showing a fourth embodiment of the invention. Inthe embodiment, a plurality of camera units 10 are arranged around thebody of a vehicle 1. Images output from those camera units 10 aresimultaneously displayed on the same screen of the information displaydevice. As shown, the camera units 10 are located at the front left L,the front right R and the back B of the vehicle 1. Each camera unitpicks up a scene within the field 10 a thereof.

Each of FIGS. 12(1) to 12(12) exemplarily shows segmented display areasformed by segmenting the screen of the information display device 4. Theimages output from the plurality of the camera units 10 aresimultaneously displayed in those display areas, respectively. Wherethose areas are used, all the images may be watched on one screen. Inthe example of FIG. 12(1), the screen is segmented in a time divisionalmanner, and only the image of the backward area is displayed. The screenis automatically changed at fixed time intervals to display otherimages. The displays may be changed manually by the driver, but use ofautomatic change of displays is suggestible since troublesome work tochange the displays is eliminated, as a matter of course.

In the example of FIG. 12(2), the screen is vertically segmented intotwo segmental display areas, right and left display areas. A front leftimage L and a front right image R are displayed in those display areas,respectively. In the example of FIG. 12(3), the screen is horizontallysegmented into two display areas, the upper and lower display areas. Theupper display area is further segmented into right and left displayareas, a front left L and a front right R. An image of the back B isdisplayed in the lower segmental display area. The example of FIG. 12(4)corresponds to the vertically inverted FIG. 12(3) example. In theexample of FIG. 12(5), images are displayed in the front left L and thefront right R, which are located in the upper portion of the displayarea which displays an image of the back B.

In the example of FIG. 12(6), the center of the upper side of a screenand both ends of the lower side of the screen are connected by twostraight lines, whereby the screen is segmented into three segmentaldisplay areas each shaped like a triangle. The front left L image isdisplayed in the upper left display area; the front right R image isdisplayed in the upper right display area; and the back B image isdisplayed in the remaining displayed area. In the example of FIG. 12(7),the center of the lower side of a screen and both ends of the upper sideof the screen are connected by two straight lines, whereby the screen issegmented into three segmental display areas each shaped like atriangle. The front left L image is displayed in the lower left displayarea; the front right R image is displayed in the lower right displayarea; and the back B image is displayed in the remaining displayed area.The example of FIG. 12(8) resembles the example of the FIG. 12(6) exceptthat the center of the upper side of the screen is connected to thecenter of the right and left sides of the screen by two straight lines.In the thus segmented screen, a display area in which the back B imageis displayed is enlarged when comparing with the corresponding one inthe FIG. 12(6) example. The example of FIG. 12(9) resembles the exampleof the FIG. 12(7) except that the center of the right and left sides ofthe screen are connected to the center of the screen area by twostraight lines. Also in this segmentation, a display area in which theback B image is displayed is enlarged when comparing with thecorresponding one in the FIG. 12(7) example.

The example of FIG. 12(10) resembles the example of the FIG. 12(8)except that the lines demarcating those segmental display areas arecurved downwardly or toward the display area of the back B image. Theexample of FIG. 12(11) resembles the example of the FIG. 12(6) exceptthat the lines demarcating those segmental display areas are curveddownwardly. The example of FIG. 12(12) resembles the example of the FIG.12(6) except that the lines demarcating those segmental display areasare curved upwardly.

While the screen of the information display device is segmented as shownin FIGS. 12(1) to 12(12) in the embodiment, it is readily understoodthat the screen may be segmented in other various ways. The displaypattern is preferably changed in accordance with running conditions ofthe vehicle. Specifically, the display pattern is automatically selectedin accordance with vehicle running conditions such that the displaypattern of FIG. 12(2) containing two segmental display areas of thefront left L and the front right R is selected for the normal running.For the backward running, any of the display patterns of FIGS. 12(3) to12(12) each containing three segmental display areas of the front leftL, front right R and back B is selected. If so done, the display willeasily understood by the driver. Alternatively, the side image displaymay be combined with the front image display or the back image displayin accordance with the turning on of the winker lamp 63 by operating thedirection indicator and a steering angle of the steering wheel.

In a display of FIG. 13, a vehicle illustration 70 is displayed on thedisplay pattern of FIG. 12(3). The display clearly shows a relativeposition of the vehicle 1 to the images from the camera units 10. Withpresence of the vehicle illustration 70, the direction of the image iseasy to understand.

It is preferable that, as shown in FIG. 14, a position of the vehicleillustration 70 is moved when the display pattern is changed to another.In the case of the time-divisional display pattern as shown in FIG.12(1), as shown in FIGS. 14(1) to 13(3), the position of the vehicleillustration 70 is moved to the upper center, lower right and lower leftpositions on the screen in accordance with the images of the back B,front left L and front right R. If so done, the driver will easilyunderstand which image is displayed. In the display pattern of FIG.12(2), as shown in FIG. 14(4), the vehicle illustration 70 is moved tothe center of the lower side of the screen. Accordingly, the driver willeasily understand that the front left L and front right R images aredisplayed.

In a display of FIG. 15, the vehicle illustration 70 is displayedtogether with a camera image direction 71 and a camera coverage 72. Fromthis display, the driver will easily understand the camera unit 10 whichpicked up the image being displayed. Further, an orientation of thevehicle illustration 70 may be changed in accordance with a steeringangle of the steering wheel 7. In this case, a relationship between theimage and the advancing direction is readily seen.

FIG. 16 shows a display pattern as shown in FIG. 12. In the display, anavigation display 41 by the navigation device 15 and an audio display42 by the audio device 16 are both displayed on the same screen in asimplified fashion. This display fully utilizes the display screen ofthe information display device 4, and is capable of effectivelyproviding information necessary for the driver of the vehicle 1 or thelike.

FIG. 17 diagrammatically shows a change of a plurality of displays bythe wipe method. In the display changing method, in a state that animage A is displayed on the entire screen of the information displaydevice, the next image B starts to shift from the left side to the rightside on the screen. Accordingly, a boundary line between the image B andthe image A moves to the right, and finally the image B occupies theentire screen. When the image A covering the entire screen area isinstantaneously changed to the image B, the driver is difficult tofollow a display change. However, in the embodiment, the driver is easyto follow the display change since the embodiment uses the wipe methodfor changing the images from one to another. The display mode change bythe wipe method is applicable to the display patterns of FIGS. 12(2) to12(12), as a matter of course.

FIG. 18 diagrammatically shows how to change a segmented area and adisplay position on a display pattern in accordance with a steeringangle of the steering wheel by the wipe method. The display pattern ofFIG. 12(3) is basically used. In the forward movement, the back B areais reduced. To turn to the left, the front left L area is increased, andto turn to the right, the front left R area is increased. In thebackward movement, the back B area is increased. With this, the imageexisting in the advancing direction is displayed while being emphasized,thereby assisting a safety check.

FIG. 19 shows a scheme of a vehicle parking assist device which is afifth embodiment of the present invention. In the embodiment, amulti-window is employed for the display by the information displaydevice 74. A touch panel is also employed. A parking assist ECU 76detects an obstructive object 47 by an obstacle sensor, such as a backsonar 17 or a corner sensor 18. When the obstructive object 47 isdetected, the parking assist ECU 76 informs the driver or the like ofthe presence of the obstructive object on the screen of the informationdisplay device.

FIGS. 20A to 20C show how to change an obstacle display by a informationdisplay device 74 in the embodiment. FIG. 20A shows a display pattern ofmulti-window presented in a normal running mode or a parking assistmode. When an obstructive object is detected, the display is changed toan obstacle display as shown in FIG. 20B or 20C. In the obstacle displayshown in FIG. 20B, a direction of an obstructive object 77 by use of avehicle illustration 70 and an arrow 78. When the obstructive object isdetected, a color of the obstacle display may be changed to call thedriver's attention. In an obstacle display shown in FIG. 20C, a displayarea for displaying an image of the camera unit 10 which picks up anobstructive object is increased relative to the other display areas. Aneasy-to-see display is presented.

FIGS. 21A and 21B show a drive assist in which a plurality of cameraunits 10 are disposed, and the images picked by the camera units 10 areseamlessly composed into a single image. Three camera units 10 areprovided for picking up a picture of a backward area of the vehicle 1.The fields 10La, 10Ba and 10Ra of the camera units partially overlapwith one another. FIG. 21B shows a display on the screen of theinformation display device 74 of the touch panel type as shown in FIG.19. In the display, the images from three camera units 10L, 10B and 10Rare seamlessly composed by expanding/reducing and cutting to form acontinuous large image.

FIGS. 22A and 22B show a display containing the images of the entirecircumferential area of the vehicle 1. As shown in FIG. 22A, six cameraunits 10F, 10FL, 10FR, 10B, 10BL, and 10BR are disposed around thevehicle 1 to pick up a scene of the entire circumferential area of thevehicle 1. The images from those camera units 10F, 10FL, 10FR, 10B,10BL, and 10BR are composed and displayed on the screen of theinformation display device 74 as shown in FIG. 22B. The driver looks thedisplay and makes a safety check on the entire circumferential area ofthe vehicle. Further, when coupled with the back sonar 17 or the cornersensor 18, it is possible to display an alarm telop or to change a colorof the display corresponding to the direction in which the obstructiveobject is detected.

In the display patterns of FIGS. 21B and 22B, a multiple of images aredisplayed on the same screen simultaneously. Accordingly, one or aplurality of images may be selectively displayed. The selection may beperformed by operating a switch by a driver or automatically performedin accordance with running conditions. Such a design as to continuouslycheck a broad area by scrolling the display is possible. The informationdisplay device 74 allows the driver to enter instructions and the likethrough the touch panel. Specifically, the driver touches a desiredimage on the display of FIG. 21B or FIG. 22B, and enlarges the selectedimage. Accordingly, the driver checks a desired location in easy andprompt manner. It is preferable to return the promptly enlarged image tothe original size. By so doing, the work by the driver to return theimage size to the original one may be eliminated.

FIG. 23 shows a scheme of a vehicle parking assist device which is aseventh embodiment of the invention. In the embodiment, a front monitorcamera 80 is attached to the front part of the body of the vehicle 1. Aroof camera 81 is also attached to the roof as the highest part of thevehicle body. A parking assist ECU 84 has an arrangement substantiallythe same as of the parking assist ECU 6 shown in FIG. 3, and it carriesout a drive assist process by the utilization of the front image pickedup by the front monitor camera 80. A navigation device 85 includes amemory for storing an image of a scene of the front area of the vehicle,which is picked up by the front monitor camera 80. This may be utilizedfor detecting traffic congestion. The fetched image is wirelesslytransmitted to a traffic control center located near a position wherethe vehicle 1 is located, by use of the navigation device 85. In thisway, a broad drive assist is performed. The image picked up by the frontmonitor camera 80 is also displayed by the information display device 4,and the driver uses it to make a safety check.

The front monitor camera 80 may also be used for a camera for detectingthe preceding vehicle. When a vehicle runs in front of the vehicle 1when viewed in the advancing direction, a vehicle-to-vehicle distancelong enough to avoid a rear-end collision must be secured. If the frontmonitor camera 80 has the function of the preceding-car detectingcamera, the drive assist and preceding-car detecting functions arerealized at low cost. The front monitor camera 80 may be used for acamera for monitoring a vehicle which forcibly runs to in front of theself vehicle. When another vehicle forcibly runs to in front of thevehicle 1, the driver must take a measure for prevent a rear-endcollision. If the front monitor camera 80 has the function to monitorsuch an interruptive vehicle, the drive assist function and the functionto detect the interruptive vehicle are realized at low cost.

The roof camera 81 picks up the front area scene from the vehicle roofas the highest position of the vehicle. When the vehicle 1 passes underan overhead bridge, the driver makes a check as to if the passing of thevehicle is permitted, when a height limit is provided. In this case, thedriver judges if the vehicle passing infringes the height limitregulation, from a view of the upper limit portion where it will pass.

FIG. 24 shows displays when a brightness of the display by theinformation display device 4, 74 is changed between the forward movementand the backward movement in each embodiment. In the backward movementshown in FIG. 24A, the entire screen is bright to provide an easy checkof the parking position 40 and the predictive traveling path curve 5 a.55 a. In the normal running shown in FIG. 24B, the screen is relativelydark. In this case, the driver must watch the front area attentively,and it is better that the driver turns his attention out of the screenof the information display device. It is for this reason that the screenis set to be relatively dark. If the screen is relatively dark, it iseasy to call driver's attention by changing the color of the screen to abright color when such a situation occurs. In the backward running, thescreen is bright so that the driver may watch the details.

FIG. 25 shows a display containing a display frame 87 of a back mirror.From the display, the driver can easily understand a back image when theback image is displayed. The back image displayed within the displayframe 87 is corrected to have a field angle of the back mirror, by acorrection image processing of cutting, enlarging/reducing, whereby thedisplay is transformed into a display fit to the driver's feeling of theimage on the back mirror. In particular when a wide angle camera isused, this correction is needed since the driver hardly grasps the senseof distance because of the image distortion.

FIG. 26 shows a basic arrangement for a drive assist in eighth to 11thembodiments of the present invention. When a driver of a vehicle 101moves backward his car to a parking zone 102 and parks the car there, avehicle drive assist device visually presents an image of a dead spacefor the driver and assists the driver in his driving on the screen of aninformation display device 104. A backward area image of the vehicle anda predictive traveling path curve 105 are displayed so that the drivercan correctly park the vehicle in a parking space marked off with whitelines 103 or the like as a target. The information display device 104may be a liquid crystal display device (LCD). The information displaydevice 104 displays a drive assist image, which is generated by aparking assist ECU 106 as an electronic control unit (ECU) for driveassist. A steerage or steering angle of a steering wheel 107 is input tothe parking assist ECU 106. In turn, the ECU 106 calculates a predictivetraveling path curve 105 based on the steering angle, and the calculatedpredictive traveling path curve 105 is displayed on the screen of theinformation display device 104.

The parking assist ECU 106 starts the supplying of drive assistinformation in response to, for example, a shifting operation of a shiftlever 108 of the transmission to an R position for backing the vehicle.The drive assist information is supplied in the form of imageinformation by the information display device 104 and acousticinformation by a speaker 109 as well.

A scene of a backward area of the vehicle 101 when the vehicle 101 backsis picked up by a camera unit 110. The camera unit 110 is installed atthe rear part of the vehicle 101 so that its field 110 a is directed tothe road 102 when the vehicle 101 moves backward. Corner sensors 111,112, 113 and 114 are provided around the vehicle 101, for example, atthe four corners of the vehicle. Those sensors serve as obstacle sensorsfor detecting obstructive objects at the corners of the vehicle by usinga ultrasonic wave.

FIG. 27 is a diagram showing a state that the vehicle 101 of FIG. 26backs into a parking space indicated by the white lines 103 in theparking zone 102. The camera unit 110 mounted on the rear part of thevehicle 101 picks up a scene of an area within the field 110 a. The areawithin the field 110 a is so selected as to contain portions, which areeach within a dead angle to the driver to the vehicle 101, as many aspossible. A steering angle of the steering wheel 107 is detected by asteering angle sensor 115. A backward movement of the vehicle isdetected as an on state of a back lamp SW signal 116 output from thetransmission.

The parking assist ECU 106 responds to a video signal of the NTSC systemfrom the camera unit 110, a pulse signal corresponding to a turn of asteering angle, which is output from the steering angle sensor 107, anda back-lamp SW signal 116 for driving a switch (SW) of a back lamp whichis turned on when the vehicle backs, and displays a predictive travelingpath curve 105 of the vehicle 101 and a three-dimensional imagegenerated along the predictive traveling path curve 105 on the screen ofthe information display device 104, thereby making a parking assist.

FIG. 28 electrically shows the innards of the parking assist ECU 106shown in FIG. 26. The parking assist ECU 106 contains a digital signalprocessor (DSP) 120 for carrying out an overall control andcalculations. Peripheral circuits are connected to the DSP 120 by a bus121. A video signal from the camera unit 110 is input in the form of acomposite video signal of the NTSC system to an amplifier/filter circuit122. The amplifier/filter circuit 122 selectively amplifies an analogvideo component contained in the composite video signal. The outputsignal of the amplifier/filter circuit is applied to an analog todigital (ADC) 123 where its analog signal form is converted into adigital form. The thus digitized video signal is then stored into afield buffer circuit 124. The output signal of the amplifier/filtercircuit 122 is also applied to a sync separation circuit 125 which inturn separates a horizontal sync signal and a vertical sync signal fromthe composite video signal. Those sync signals are input to the DSP 120.An angular displacement detect signal, output from the steering anglesensor 115, which represents a steering angle of the steering wheel 107,is input to the DSP 120 by way of a buffer circuit 126. A centerposition detect signal generated when the center position of thesteering wheel 107 is detected is also input to the DSP 120 through thebuffer circuit 126. Further, obstacle detect signals derived from thecorner sensors 111 to 114, and a back lamp SW signal are also input theDSP 120, through the buffer circuit 126.

The DSP 120 operates in accordance with a program prestored in theprogram memory 127, which is connected thereto by the bus 121. Datanecessary for an operation based on the program of the program memory127 is stored in advance in a data memory 128. The DSP 120 carries out arecognition process of the white lines 103 and the like in response tothe video signal input thereto, and calculations for generating apredictive traveling path curve 105 according to an angular displacementoperation of the steering wheel 107. The generated image is stored intofield buffers 131 and 132, either of which is selected according to anoutput signal of an SW circuit 130. Either of the field buffers 131 and132 is selected by the SW circuit 130, and an output signal of the SWcircuit is output as an NTSC video signal to the information displaydevice 104, by way of a digital to analog converter (DAC) 133 and afilter/amplifier circuit 134. Electric power is supplied from a powersupply 135 to the whole parking assist ECU 106. A reset circuit 136supplies a reset signal to the parking assist ECU 106 to reset thelatter to an initial state, and starts its operation according to theprogram. A CLK/frequency demultiplying circuit 137 supplies a clocksignal defining a reference timing of the operation of the parkingassist ECU 106, and signals formed by demultiplying a frequency of theclock signal.

FIG. 29 is a diagram roughly showing a display of a parking assist imagein the eighth embodiment of the invention. In the embodiment,rectangular planes 140, 141, 142, . . . , which depends on the height ofthe vehicle, are arrayed at fixed distances (e.g., 1 m) along apredictive traveling path curve 105 in the display. Of those planes 140,141, 142, . . . , the plane a41 is based on a case where the vehicleapproaches to an obstructive object sensed by the corner sensors 111 and112 of the rear part of the vehicle, and it has a color different fromthose of other planes 140 and 142.

FIG. 30 is a flow chart showing a drive assist procedure executed by theparking assist ECU 106 in the FIG. 29 embodiment. The drive assistprocedure starts at a step a1. In a step a2, the parking assist ECU 106,more exactly DSP 120, checks if a parking assist mode is set up. Whenthe back lamp SW signal 116 input thereto is in an on state, the DSPjudges that the parking assist mode is set up. In a step a3, a steeringangle is detected from an angular displacement detection signal of thesteering angle sensor 115. In a step a4, a predictive traveling pathcurve 105 is calculated from a steering angle. In a step a5, thecalculated predictive traveling path curve 105 is superimposed on thebackward area image output from the camera unit 110. In a step a6, anobstacle detecting operation by the corner sensors 111 and 112 attachedto the rear part of the vehicle is carried out. The corner sensors 111and 112 radiate ultrasonic waves toward an obstructive object, and anobstructive object is detected on the basis of presence or absence ofultrasonic waves from the obstructive object. A distance from thevehicle to the obstructive object is measured on the basis of a timeduration from an instant that the ultrasonic wave is transmitted till itis received. In a step a7, distances for presenting the planes 140, 141and 142 as shown in FIG. 28 are initialized.

In a step a8, a fixed distance “d”, e.g., 1 m is added to the distance.In a step a9, one of the rectangular planes 140, 141, 142, . . . isdisplayed on the predictive traveling path curve corresponding to thedistance. Those planes 140, 141, 142, . . . are displayed vertical tothe ground. A step a10 checks if the positions of the planes 140, 141,142, . . . are close to the position of the obstructive object 150 whichis detected in the step a6. When the positions are close to the obstacleposition, a step all changes the colors of the planes 140, 141, 142, . .. to be displayed from those of the planes 140, 141, 142, . . . at otherpositions to other ones. When those plane positions are not close to theobstacle position in the step a10, or when the color change in the stepall is completed, a step a12 checks if the distance reaches the finalposition. The final position of the distance is set in advance. When thedistance does not reach the final position, the DSP returns to the stepa8 in its procedure execution. When the step a12 judges that thedistance reaches the final position, the DSP returns to the step a2, andrepeats a sequence of procedure steps a2 to a12.

When completing the parking operation by backing the vehicle, the drivershifts the shift lever 108 to a P position for parking. Accordingly, theback lamp SW signal 116 is in an off state, and the parking assist modeterminates. When the step a2 judges that the vehicle is not in theparking assist mode, the parking assist operation ends in a step a2.

FIG. 31 is a diagram showing an image for parking assist which is theninth embodiment of the invention. In this embodiment, a plane 140displayed along the predictive traveling path curve 105 is displayed ata position where the obstructive object 150 is detected. The plane 140is a rectangular in shape, and its height corresponds to the height ofthe vehicle 101. Therefore, the height of the vehicle 101 relative tothe obstructive object 150 can be recognized, and hence the driver cancheck a possibility that his vehicle will touch the obstructive object150 and a touching position when it touches the object.

FIG. 32 is a flow chart showing a control procedure executed by theparking assist ECU 106 in the FIG. 31 embodiment. The control procedurestarts at a step b1. Steps b1 to b6 correspond to the steps a1 to a6 inFIG. 30. In a step b7, a plane 140 is displayed on the predictivetraveling path curve 105 located close to the obstructive object 150detected in the step b6. When the plane display in the step b7 ends, theDSP returns to the step b2 in its procedure execution, and repeats asequence of the steps b2 to b7. When the step b2 judges that the parkingassist mode ends, the DSP completes the procedure control in step b8.

FIG. 33 is a diagram showing a parking assist image which forms thetenth embodiment of the invention. In the embodiment, the rectangularplane 140 of which the height corresponds to that of the vehicle movesalong the predictive traveling path curve 105. The color of the plane140 is changed at a position close to the obstructive object 150 fromthat of the plane at other positions. Also in the embodiment, the heightof the vehicle 101 can be recognized, and a possibility of the backingvehicle with the obstructive object 150 in the parking space can bechecked.

FIG. 34 is a flow chart showing a control procedure of the parkingassist ECU 106 in the FIG. 33 embodiment. The control procedurecommences at a step c1. A control sequence from steps c1 to c7 isequivalent to the control sequence from the steps a1 to a7 in FIG. 32.In a step c8, a distance Δd as a distance unit of the plane 140 in thedisplay is added to the distance. A value of the unit distance Δd isselected to be smaller than the fixed distance “d” in the FIG. 28embodiment. In a step c9, a plane 140 is displayed at the distanceposition. A step c10 checks if the position at which the plane 140 isclose to the obstructive object 150. When the answer is YES, a step c11changes the color of the plane 140 to another color. When the answer isNO in the step c10, or when the color of the plane 140 is changed in thestep c11, a step c12 checks if it reaches the final position. At thisfinal position is equivalent to that judged in the step a12 in FIG. 30.In a step c13, the DSP deletes the plane 140 displayed in the step c13,and returns to the step c8.

Subsequently, the plane 140 is displayed every unit distance Δd, and ismoved along the predictive traveling path curve 105 till a step c12judges that it reaches the final position. When judgement is made suchthat the vehicle is close to the obstructive object 150 during itsmovement, the color of the plane 140 is changed. When the step c12judges that it reaches the final position, the DSP returns to the stepc2, and repeats a procedure sequence from the steps c2 to c13. When thestep c2 judges that the parking assist mode ends, the control procedureends in a step c14.

In the FIG. 33 embodiment, the color of the plane 140 is changed at aposition near the obstructive object 150. If required, the plane 140 maybe stopped at a position close to the obstructive object 150. Thedisplay of the plane 140 may be stopped in a manner that the movement ofthe plane 140 is stopped in the step c11 of FIG. 34, in place of thechanging the color of the plane 140. In each of the FIGS. 29, 31 and 33embodiments, one or more number of the rectangular planes 140, 141, 142,. . . which depend on the vehicle height. Alternatively, a projectionplane of the vehicle may be displayed vertically to the ground. Wherethe projective plane of the vehicle is displayed, more practical checkis made about the possibility of the touching of the vehicle with theobstructive object.

FIG. 35 is a diagram showing a parking assist display as the 11thembodiment of the invention. In the embodiment, a backward movingvehicle 101 is illustrated in a 3-dimensional simulation. A simulationimage 151 of the vehicle 101 is generated according to a configurationof the vehicle 101 stored in the data memory 128. When the parkingassist display is used, the driver can check, in a simulation manner, apossibility that the vehicle will touch an obstructive object locatednear the parking space while moving the vehicle along the predictivetraveling path curve 105.

FIG. 36 is a flow chart showing a control procedure of the parkingassist ECU 106 for the FIG. 35 embodiment. The control procedurecommences at a step d1, and a procedure sequence from steps d1 to d8 isequivalent to that from the steps c1 to c8 in FIG. 34. In a step d9, asimulation image 151 is displayed in place of the plane 140 in FIG. 30.A subsequent procedure sequence to a step d13 is equivalent to thesequence from the steps c9 to c13 in FIG. 34, except that the sequenceis directed to the simulation image 151 in place of the plane 140.

In the embodiments described above, the corner sensors 111 and 112 areused for the obstacle sensors. Those sensors may be substituted by aback sonar using ultrasonic wave, a radar using electromangetic wave orlaser beam. An obstructive object may be extracted from the image pickedup by the camera unit 110 by use of the image processing technique. Inthe embodiments, the colors of the planes 140, 141, 142, . . . and thesimulation image 151 are changed at a position closer to the obstructiveobject 150. Instead of the color change, flashing may be used fordistinguishing it from other positions.

While the parking assist has been described when the vehicle 101 isbacked into the parking space, the same thing is correspondingly appliedto the parking assist when the vehicle 101 is advanced into the parkingspace. It is evident that the technical idea of each embodiment may beapplied to the parking assist when the vehicle travels in a path wherethe visibility is bad.

FIG. 37 is a diagram showing a vehicle 201 under drive assist which ismoving to a parking space in a 12th embodiment of the present invention.When the vehicle 201 backs to a parking zone 202 and is to be parked ina parking space marked off with white lines 203, image information forparking assist is displayed on the screen of an information displaydevice 204. The parking assist image information contains a predictivetraveling path curve 205 which is calculated on the basis of theprediction of a traveling path of the vehicle. The predictive travelingpath curve 205 is calculated depending on a steerage or steering angleof a turn of a steering wheel 207 of the vehicle 201, by a parkingassist ECU 206 as an electronic control unit for parking assist. Theparking assist ECU 206 starts a control for drive assist when a shiftlever 208 of the transmission is shifted to a reverse gear position, forexample. A guide for parking assist in the parking operation is alsoacoustically carried out by use of a speaker 209.

When the driver backs his vehicle into a parking space, a backward areaof the vehicle is the dead space when viewed from the driver of thevehicle 201 and hence he insufficiently sees the background area. Tocope with this, a camera unit 210 is attached to the upper rear part ofthe body of the vehicle 201 so that its field 210 a covers the parkingzone 202 which forms a dead angle for the driver. A scene of the parkingzone 202 is picked up by the camera unit and displayed on the screen ofthe information display device 204.

FIG. 38 is an electrical arrangement when the driver backs the vehicle201 of FIG. 37 to the parking zone 202 and parks it there. A videosignal of the NTSC type is input from the camera unit 210 to the parkingassist ECU 206. A pulse signal is also input to the parking assist ECUevery fixed steering angle, from a steering angle sensor 211 fordetecting a steering angle of a steering wheel 207. When the shift lever208 is shifted to the reverse gear position, a back-lamp SW signal 212which turns on a switch (SW) to turn on a back lamp is produced andinput to the parking assist ECU 206. When the back-lamp SW signal 212 isput to an on state, the parking assist ECU 206 starts a parking assistnecessary for the vehicle to back into a parking space. In the parkingassist operation, a predictive traveling path curve 205 which depends ona steering angle detected by the steering angle sensor 211 iscalculated, and overlaid on the image of the backward area picked up bythe camera unit 210 on the screen of the information display device 204.

FIG. 39 is an electrical arrangement of the parking assist ECU 206 shownin FIG. 37. In the parking assist ECU 206, an overall control isperformed by a digital signal processor (DSP) 220. The DSP 220 performscontrols and signal processings through a bus 221. The camera unit 210inputs a video signals of the NTSC type to the parking assist ECU. Thevideo signal is amplified by an amplifier/filter circuit 222. A signalform of the vide component signal is converted into A digital form by andigital converter (ADC) 223. The digitized video signal is stored in afield buffer circuit 224. The video signal is also applied from theamplifier/filter circuit 222 to a sync separation circuit 225. In thesync separation circuit, the horizontal and vertical sync signals areseparated from the video signal and are input the DSP 220. An angulardetect signal representative of an angular displacement of the steeringwheel is applied to the DSP 220, from the steering angle sensor 211. Acenter position detect signal indicating that the steering wheel 207 isat the center position is also applied to the DSP 220, through the bus221.

The DSP 220 operates in accordance with a program prestored in theprogram memory 227 connected thereto by way of the bus 221. Undercontrol of the program, the DSP 220 carries out calculation processesand performs control operations, while referring to data preset in thedata memory 228 and storing data produced during the calculation. TheDSP 220 recognizes the white lines 203 shown in FIG. 37 in accordancewith the video signal input thereto and generates an image of apredictive traveling path curve 205, which depends on a steering angleof the steering angle sensor 211.

The generated image is stored into field buffers 231 and 232 each ofwhich is selected by a SW circuit 230. The image signal is selected bythe SW circuit 230, and the selected one passes through a digital toanalog converter (DAC) 233 and a filter/amplifier circuit 234, and goesas a video output signal to the information display device 204. Electricpower is supplied from a power supply 235 to the whole parking assistECU 206. The DSP 220 receives a reset signal to the DSP from a resetcircuit 236, and a clock signal which defines a basic operation timingfrom a CLK/frequency demultiplying circuit 237 and signals having thefrequencies produced by demultiplying the frequency of the clock signal.

The parking assist ECU 206 includes a voice data memory 238 and a voicecomposing circuit 239. With provision of those, the parking assist ECU206 carries out a drive guide by a voice. To the voice guide, the DSP220 selects voice data from those stored in the voice data memory 238 inaccordance with guide contents, and applies the selected one to thevoice composing circuit 239. The voice data applied to the voicecomposing circuit 239 is digital data encoded according to apredetermined encoding method. The voice composing circuit 239 decodesthe voice data and converts it into an analog voice signal. The analogvoice signal output from the voice composing circuit 239 is output in anacoustic form by a speaker 209, whereby the vehicle drive assist deviceacoustically assists the driver in his driving.

FIG. 40 is a flow chart showing a control procedure for parking assist,which is carried out by the parking assist ECU 206 in the 12thembodiment. The control procedure commences at a step a1. In a step s2,the parking assist ECU 206, more exactly the DSP 220, checks if aback-lamp SW signal 212 is in an on state. When the back-lamp SW signal212 is in an on state in the step s2, a parking assist mode is started.In a step s3, an alarm of causing the driver to visually watchenvironmental conditions around the vehicle is issued acoustically. Abackward area scene of the vehicle 201 is picked up by the camera unit210 and displayed on the screen of the information display device 204.The driver is required to watch attentively the front and side areas ofthe vehicle 201, which are not displayed by the information displaydevice 204. The acoustic alarm is given to prevent the driver to watchonly the screen of the information display device 204. An example of theacoustic alarm is a sound message “Watch the environment in backing thecar”.

In the next step s4, the backward area image picked up by the cameraunit 210 is displayed on the screen of the information display device204, and in a step s5, a steering angle is detected by an angulardisplacement detect signal derived from the steering angle sensor 211.In a step s6, a predictive traveling path curve 205 is calculateddepending on a steering angle detected, and is overlaid on the backwardarea image on the screen of the information display device 204. In astep s7, a target parking position is detected from the image picked upby the camera unit 210. It is required that the target parking positiondetected meets the conditions that it is the parking space marked offwith the white lines 203, and that it has an area large enough to parkthe vehicle.

In a step s8, the DSP checks if the target parking position detected inthe step s7 lies on the predictive traveling path curve, which iscalculated in the step s6 depending on a steering angle detected in thestep s5. When the target parking position is not on the predictivetraveling path curve, a step s9 calculates a target traveling path fromthe present position to the target parking position. The DSP judges if asteering angle that will produce the target traveling path is within arange within which the steering wheel 207 may be turned, on the basis ofa difference between the target traveling path and the actual predictivetraveling path curve 205. When it is judged that the steering angle maybe adjusted by varying the steering angle so as to satisfy the targettraveling path, a step s10 carries out a steering guide about adirection and a angle of a turn of the steering wheel 207 on the basisof a steering angle calculated for matching an actual steering angle tothe target traveling path. The steering guide may take the form of adisplay of character information, e.g., telop on the screen of theinformation display device 204 or a voice message “turn more to theright (left)”. Following the step s10 of the steering guide, the DSPreturns to the step s5 in execution of the control procedure. When thedriver varies the steering angle, the drive assist for the steeringguide is performed through a sequence of the steps s5 to s10 till thesteering angle to produce the target traveling path is reached.

When the step s8 judges that the target parking position lies on thepredictive traveling path curve, a step s11 judges that the vehicle willtouch an obstructive object when the vehicle 201 backs along thepredictive traveling path curve 205. When the vehicle does not touch theobstructive object, a step s12 judges if a parking stopper exists in thetarget parking position. If the stopper exists, a step s13 presents aguide that the stopper is preset by a display by the information displaydevice 204 or by a voice. Thus, when the driver is informed of presenceor absence of the parking stopper, he may refer to it in his brakingoperation when he backs the vehicle.

When the step s9 judges that the vehicle cannot run into the targetparking position by the steering operation or when the step s11 judgesthat the vehicle will touch the obstructive object on the predictivetraveling path curve, a step s14 presents a guide to reverse theadvancing direction of the vehicle. The guide of reversing the advancingdirection is given by a display by the information display device 204 ora voice. The guide is such that you cannot park the vehicle if thevehicle will be moved backward, and to park the vehicle, you must movethe vehicle forward and reversely turn the steering wheel 207.

When the step s2 judges that the back-lamp SW signal 212 is not in an onstate, when the step s12 judges that the parking stopper is not present,or when the guide ends in the step s13 or s14, the control procedureends at a step s15. When the driver moves his vehicle forward after thestep s14 of the advancing-direction reverse guide, the DSP continues theoperation of detecting the target parking position on the backward areaimage, and calculates a target traveling path from the present positionto the target parking position. The guide may be made to inform thedriver of the timing of reversely turning the steering wheel 207 when itis confirmed that the time to produce the target traveling path at asteering angle through which the steering wheel 207 may be turned isreached and the vehicle does not touch the obstructive object on thetarget traveling path.

While the parking assist has been described for a case where the driverbacks the vehicle into the parking zone 202, the parking assist may becorrespondingly applied to a case where the vehicle 201 moves forwardinto the parking lot. In this case, a scene of the forward area of thevehicle is picked up, and a target parking position is detected. In the12th embodiment mentioned above, the steering guide, stopperpresence/absence guide and the advancing-direction guide are carried outin the steps s10, 13 and 14, respectively. If occasion demands, one ofthose guides or a combination of them may selectively be used.

While a display by the information display device 204 and a voice by thespeaker 209 are both used for the guide for the parking assist in the12th embodiment mentioned above, one of the visual and acoustic guidesmay be used, as a matter of course.

FIG. 41 shows a basic arrangement for a drive assist in 13th to 18thembodiments of the present invention. When a driver of vehicle 301 amoves backward his car to a parking zone 302 and parks the car there, avehicle parking assist device visually presents an image in a dead spacefor the driver and assists the driver in his driving on the screen of aninformation display device 304. A backward area image and a predictivetraveling path curve 305 are also displayed so that the driver cancorrectly park the vehicle into a parking space marked off with whitelines 303 or the like as a target. The information display device 304may be a liquid crystal display device (LCD). The information displaydevice 304 displays a drive assist image, which is generated by aparking assist ECU 306 as an electronic control unit (ECU) for driveassist. A steerage or steering angle of a steering wheel 307 is input tothe parking assist ECU 306. In turn, the ECU 306 calculates a predictivetraveling path curve 305 based on the steering angle, and the calculatedpredictive traveling path curve 305 is displayed on the screen of theinformation display device 304.

The parking assist ECU 306 starts the supplying of drive assistinformation in response to, for example, a shifting operation of a shiftlever 308 of the transmission to an R position for moving the vehiclebackward. The drive assist information is supplied in the form of visualor image information by the information display device 304 and acousticinformation by a speaker 309 as well. A scene of a backward area of thevehicle 301 when the vehicle 301 backs is picked up by a camera unit310. The camera unit 310 is installed at the rear part of the vehicle301 so that its field 310 a is directed to the parking zone 302 when thevehicle 301 moves backward. A steering angle of the steering wheel 307is detected by a steering angle sensor 11. A shifting operation of theshift lever 308 to the reverse gear position is judged according to aback-lamp SW signal 312 as an on/off signal applied to a switch(referred to as an SW) for controlling a lighting condition of a backlamp.

FIG. 42 is a diagram showing a state that the vehicle 301 of FIG. 41backs to a parking space marked off with white lines 303 in the parkingzone 302. The camera unit 310 mounted on the rear part of the vehicle301 picks up a scene of an area within the field 310 a. The area withinthe field 310 a is so selected as to contain portions, which are eachwithin a dead angle to the driver to the vehicle 301, as many aspossible. The parking assist ECU 306 also receives a pulse signalcorresponding to a steering angle detected by a steering angle sensor311, and a back-lamp SW signal 312, which is output from thetransmission and put in an on state when the vehicle is backing.

The parking assist ECU 306 responds to a video signal of the NTSC systemfrom the camera unit 310, a pulse signal corresponding to a turn of asteering angle, which is output from the steering angle sensor 311, anda back-lamp SW signal 312 of the back lamp which is turned on when thevehicle backs, and displays a predictive traveling path curve 305 of thevehicle 301 and a three-dimensional image generated along the predictivetraveling path curve 305 on the screen of the information display device304, thereby making a parking assist.

FIG. 43 electrically shows the innards of the parking assist ECU 306shown in FIG. 41. The parking assist ECU 306 contains a digital signalprocessor (DSP) 320 for carrying out an overall control andcalculations. Peripheral circuits are connected to the DSP 320 by a bus321. A video signal from the camera unit 310 is input in the form of acomposite video signal of the NTSC system to an amplifier/filter circuit322. The amplifier/filter circuit 322 selectively amplifies an analogvideo component contained in the composite video signal. The outputsignal of the amplifier/filter circuit is applied to an analog todigital (ADC) 323 where its analog signal form is converted into adigital form. The thus digitized video signal is then stored into afield buffer circuit 324. The output signal of the amplifier/filtercircuit 322 is also applied to a sync separation circuit 325 which inturn separates a horizontal sync signal and a vertical sync signal fromthe composite video signal. Those sync signals are input to the DSP 320.An angular displacement detect signal output from the steering anglesensor 311 and a center position detect signal generated when the centerposition of the steering wheel 307 is detected are also input to the DSP320. A back-lamp signal is also applied to the DSP 320 by way of abuffer circuit 326.

The DSP 320 operates in accordance with a program prestored in theprogram memory 327, which is connected thereto by the bus 321. Datanecessary for an operation based on the program of the program memory327 is stored in advance in a data memory 328. The DSP 320 carries out arecognition process of the white lines 303 and the like in response tothe video signal input thereto, and calculations for generating apredictive traveling path curve 305 based on an angular displacementoperation of the steering wheel 307.

The generated image is stored into field buffers 331 and 332, either ofwhich is selected according to an output signal of an SW circuit 30.Either of the field buffers 331 and 332 is selected by the SW circuit330, and an output signal of the SW circuit is output as an NTSC videosignal to the information display device 304, by way of a digital toanalog converter (DAC) 333 and a filter/amplifier circuit 334. Electricpower is supplied from a power supply 335 to the whole parking assistECU 306. A reset circuit 336 supplies a reset signal to the parkingassist ECU 306 to reset the latter to an initial state, and starts itsoperation according to the program. A CLK/frequency demultiplyingcircuit 337 supplies a clock signal defining a reference timing of theoperation of the parking assist ECU 306, and signals formed bydemultiplying a frequency of the clock signal.

FIG. 44 is a diagram roughly showing a display of a parking assist imagein the 13th embodiment of the invention. In the 13th embodiment, guidelines 341 and 342, and distance lines 341, 342 and 345 are displayed inthe backward area image picked by the camera unit 310 so as to providean easy recognition of distance when an obstructive object, e.g., aparking stopper, is present. The guide lines 341 and 342 are rectilinearlines extending backward from both sides of the vehicle 301, and thedistance lines 343 to 345 are lines for indicating distances from therear end of the vehicle. A width of a space between the guide lines 341and 342 corresponds to the width of the vehicle. Therefore, if thoselines guide lines are parallel to the white lines 303 demarcating theparking space, all the driver has to do is to merely move back thevehicle straightly along the guide lines 341 and 342. Thus, the displayof the FIG. 44 image makes it easy for the driver to easily grasp arough distance to the obstructive object 340 when moving back thevehicle, and hence he easily sets the stop position.

FIG. 45 is a diagram roughly showing a display of a parking assist imagein the 14th embodiment of the invention. In the 14th embodiment, alength of the predictive traveling path curve 305 is adjusted inaccordance with a distance ranging to the parking position. The parkingposition may be set at a position of an obstructive object 340, such asa parking stopper. Accordingly, a length of the predictive travelingpath curve 305 that is overlaid on the backward area image of thevehicle 301 is a distance from the rear end of the vehicle 301 to theobstructive object 340. Thus, the length of the predictive travelingpath curve 305 may be adjusted in accordance with a distance to theparking position. Accordingly, an easy-to-understand display ispresented to the driver.

FIG. 46 is a diagram roughly showing a display of a parking assist imagein the 15th embodiment of the invention. In the 15th embodiment, it isassumed that three parking spaces 350, 351 and 352 are present side byside. The parking spaces 350 and 352, located on both sides of theparking space 351, are already occupied by other vehicles 353 and 354.When the moving vehicle 301 backs to the empty parking space 351 locatedbetween those vehicles 353 and 354, a roof predictive traveling pathcurve 355 is displayed to avoid colliding of the vehicle 301 with theroofs of the vehicles 353 and 354.

The vehicle 301 takes a three-dimensional configuration. When thepredictive traveling path curve 305 corresponding to the bottom of thevehicle 301 is used, it is difficult to judge whether or not the roofportion of the self vehicle or the like will hit another vehicle 353 or354 or an obstructive object at a portion of the self vehicle which isshaded by the adjacent vehicle 353 and forms a dead space to the driver.In this case, when the roof predictive traveling path curve 355 is used,the driver is successfully guided to the intended parking space in astate that the self vehicle does not hit the roof of the other vehicle353 or 354.

FIG. 47 is a diagram roughly showing a display of a parking assist imagein the 16th embodiment of the invention. Also in the 16th embodiment, asin the FIG. 46 embodiment, it is assumed that three parking spaces 350to 351 are present, and of those parking spaces, two parking spaces 350and 351 are already occupied by other cars 353 and 354. Projection lines356 and 357 of the vehicles 353 and 354 to the ground are displayed. Bydisplaying the projective lines 56 and 57 of the vehicles 350 and 351parked in the parking spaces 350 and 351 located on both sides of theparking space 351, the vehicle 301 can accurately be moved into theparking space 351 irrespective of, in particular, the side portion ofthe vehicle 353 parked on this side of the parking space 351, whichforms a dead space to the driver.

FIG. 48 is a diagram roughly showing a display of a parking assist imagein the 17th embodiment of the invention. Also in the 17th embodiment, asin the FIGS. 46 and 47 embodiments, it is assumed that three parkingspaces 350 to 351 are present, and of those parking spaces, two parkingspaces 350 and 351 are already occupied by other cars 353 and 354. Inthe 17th embodiment, a layout of other vehicles 353 and 354 parked sideby side in the parking spaces on both sides of the parking space 351 anda self vehicle 301, when viewed from the top, is modeled and displayedin the form of a plane model 380. The plane model 380 is generated, bythe DSP 320, on the basis of the rough configurations of the parkingspaces 350 to 351 and the other vehicles 353 and 354, positions of thevehicle 301 relative to other things, and a configuration of the vehicle301 that is already input. With the display of the vehicle 301 locatedrelative to other vehicles and the like as viewed from the top, thedriver can easily grasp a position of the self vehicle relative to theparking space 351 and the other vehicles 353 and 354 parked in theparking space between the parking space 351, and an orientation andadvancing direction of the self vehicle 301.

Also in the plane display shown in FIG. 48, a direction in which theself vehicle 301 backs is oriented downward on the screen of theinformation display device 304. The plane display containing the thusoriented vehicle matches such a driver's feeling that he feels quietafter parking his car in the parking space 351. For the plane model 380,if a parking zone is the regularly or frequently used one, the shapesand the like of the parking spaces 350 to 351 may be input in the formof data in advance.

FIG. 49 is a block diagram showing an electrical arrangement of aparking assist ECU 366 used in the 18th embodiment of the invention. Theparking assist ECU 366 resembles the parking assist ECU 306 shown inFIG. 43, and hence like or equivalent portions in the figure aredesignated by like reference numerals in FIG. 43, for simplicity ofexplanation. An angular displacement detect signal and a center positionsignal, and a shift lever position signal representing a gear positionof the shift lever 308 and a signal derived from the select switch 367are input to the DSP 320 by way of a buffer circuit 368. A timer 369 isalso connected to the bus 321. The timer 369 may be operated to count atime set by the DSP 320.

The select switch 367 is used for selecting a display of the plane model380 as shown in FIG. 48 or a normal display of the backward area imageas shown in FIG. 46 or 47. A driver of the vehicle 301 can appropriatelyselect one of those displays according to parking space conditions byoperating the select switch 337.

FIG. 50 shows a parking assist procedural process in the FIG. 49embodiment. The parking assist procedure starts at a step s1. In a steps2, what executes the procedural process, or the DSP 320, checks if theshift lever 308 is shifted to the reverse gear position, by use of ashift lever position signal. If it is at the reverse gear position, theDSP 320 judges that a parking assist mode is set up, and executes a steps3. In this step, the DSP 320 checks if the select switch 367 is turnedto a position of the plane model 380. When it is turned to the planemodel position, a step s4 is executed to display a plane model 380. Whenthe select switch 367 is not turned to the plane model position in thestep s3, a step s5 is executed to display a backward area image.

After the step s4 or s5 ends in its execution, a step s6 is executed tocheck if the shift lever 308 is shifted to a drive gear position. If itis not turned to the drive position, the DSP 320 considers that thebackward movement for parking continues, and returns to the step s3.Subsequently, a sequence of the process execution from the step s3 tothe step a6 is repeated.

When the shift lever 308 is shifted to the drive gear position, thevehicle advances. In the present embodiment, the parking assist iscontinued for a fixed time so as to perform an effective parking assistalso when after moving back the vehicle, the driver slightly moves itforward for a reverse turn of the steering wheel, and then moves back itfor parking. To this end, in a step s7, the counting operation of thetimer 369 is started, and the parking assist is continued till thecounting operation by the timer 369 ends in a step s8. When the countingoperation by the timer 369 ends in the step s8 or the shift lever 308 isnot at the reverse gear position in the step s2, the procedural processends in a step s9. A fixed time, e.g., several tens seconds, is set inadvance in the timer 369. When the shift lever 308 is shifted to thereverse gear position before this set time elapses, the DSP 320 may bereturned to the step s3.

In each of the 13th to 18th embodiments mentioned above, the DSP 320 isused for the calculating process and control in the parking assist ECU306 (366). If required, a general CPU may be used instead of the DSP.The DSPs or CPUs may be combined in use. For the image processing in theinvention, a semiconductor integrated circuit designed to be usedexclusively for the image processing may be used.

FIG. 51 shows an external appearance of a vehicle or vehicle 401 as anobject which will undergo a series parking assist in 19th and 20thembodiments of the present invention which will be described hereunder.When the vehicle 401 moves on and along a traffic lane demarcated withwhite lines 403 on a road 402 and backs from the traffic lane to theshoulder edge of the road and parks in a location close to and along theshoulder edge of the road, image information for the series parkingassistance is displayed on the screen of an information display device404. The image information contain a predictive traveling path curve 405depicted on the calculation based on a prediction of a traveling path ofthe vehicle 401, and others. The predictive traveling path curve 405 iscalculated using a steering angle as an operation angle of a steeringwheel 407 for steering the vehicle 401, by a parking assist ECU 406 asan electronic control unit (referred to as “ECU”) for making a driveassist for the series parking. When a shift lever 408 of thetransmission is shifted to the reverse gear position, for example, and aleft winker lamp 409L of those right and left winker lamps 409L and 409Rlights up, the display starts to assist a driver in parking the vehiclein a parking space along the left side of the load.

In moving the vehicle 401 backward for its parking, a backward area ofthe vehicle 401 is the dead space when viewed from the driver, and hecannot sufficiently watch a backward area of the vehicle. To cope withthis, a camera unit 410 is attached to the upper part of the body of thevehicle 401, and picks up a scene of the backward area defined by afield 410 a. The camera unit 410 produces a video signal of the NTSCtype, for example, and the parking assist ECU 406 displays a predictivetraveling path curve 405 overlaid on the backward area image on thescreen of the display device 404.

FIG. 52 is a diagrammatic illustration showing a basic arrangement for adrive assist when the driver starts a series parking of the vehicle 401of FIG. 51. A video signal input from the camera unit 410 to the parkingassist ECU 406 is of the NTSC type. A steering angle of the steeringwheel 407 is detected by a steering sensor 411. The steering sensor 411produces a pulse signal every angular displacement of the steering wheel407 at a fixed angle, in the form of an angular displacement signal.When the shift lever 408 of the transmission of the vehicle 401 isoperated to the reverse gear position, a back lamp SW signal 412 forturning on a switch (referred to as a SW) to light up a back lamp isproduced and input to the parking assist ECU 406. A light-up signal forthe winder lamps 409L and 409R is also input to the parking assist ECU406. An image of a parking space 413 is displayed on the screen of thedisplay device 404. At this time, a shoulder edge of the road 402 isalso displayed. The parking space 413 is located between, or example,vehicles 415 and 416 preceding and subsequent to the self-vehicle.

FIG. 53 schematically shows an electrical arrangement of the parkingassist ECU 406 shown in FIG. 51. An overall control in the parkingassist ECU 406 is executed by a digital signal processor (DSP) 420. TheDSP 420 executes signal processing through a bus 421. The camera unit410 produces an NTSC video signal, and the video signal is amplified byan amplifier/filter circuit 422. The amplifier/filter circuit 422extracts a video component signal from the video signal, and a signalform of the video component signal is converted from an analog form to adigital form by an analog to digital converter (referred to as an ADC)423. The digitized video component signal is stored into a field buffercircuit 424. The amplifier/filter circuit 422 also applies a videosignal to a sync separation circuit 425 where horizontal and verticalsync components are separated from the video signal. And those syncsignals are input to the DSP 420. An angular displacement detect signal,which is representative of an angular displacement of a steering angle,and a center position signal, which represents that the steering wheel407 is at the center position, are input to the DSP 420 by way of abuffer circuit 426. A back lamp SW signal and a window lamp light-upsignal are also input to the DSP 420 via the buffer circuit 426. The DSP420 operates in accordance with a program that is stored in advance in aprogram memory 427, which is connected via the bus 421 to the DSP 420per se. The DSP 420 is capable of reading out data necessary for theprogrammed operation form a data memory 428, and writes data into thesame. The DSP 420 recognizes, in accordance with an incoming videsignal, the white lines 403 shown in FIG. 51, the shoulder edge of theroad 402, and other parking vehicles. Further, it is also capable ofcalculating, on the basis of the recognition result, a target parkingposition of the vehicle 401, and an ideal traveling path or path curvealong which the vehicle moves from the present position to the targetparking position.

The DSP 420 generates it so that it is overlaid on the image derivedfrom the camera unit 410, and stores them into field buffers 431 and 432the outputs of which are selectively output through a SW circuit 430.The output signal of each of the field buffers 431 and 432, which isselected by the SW circuit 430, is output as a video output signal tothe display device 404, through a digital to analog (DAC) 433 and afilter/amplifier circuit 434. Electric power is fed from a power supply435 to the whole parking assist ECU 406. The parking assist ECU alsoreceives a reset signal from a reset circuit 436, and a clock signaldefining a base for the operation timings and signals of demultipliedfrequencies from a CLK/frequency demultiplying circuit 437.

FIG. 54 shows, in a simplified form, an image display presented on thescreen of the information display device 404, which the image display iscarried out as the 19th embodiment of the present invention. A scene ofthe parking space 413 picked by the camera unit 410 is displayed on thescreen of the display device 404. Other vehicles 415 and 416 and thelike are present before and after the parking space 413. A predictivetraveling path curve 405, which is calculated depending on the presentsteering angle, is displayed extending from the rear end of the vehicle401 to the parking space 413. To properly park the vehicle 401 in theparking space 413, the steering wheel 407 must be reversely turnedduring the backward movement. The parking assist ECU 406 as timingjudging means determines a vehicle position suitable for the reverseturn of the steering wheel 407 on the predictive traveling path curve405, and draws a guide line 440 of an oblique line at the suitableposition. The guide line 440 is drawn at a position varying with thebackward movement of the vehicle 401 within an area within which thevehicle 401 may be parked at a target parking position 445 in theparking space 413 by operating the steering wheel 407. When the driverof the vehicle 401 reversely turns the steering wheel 407 at a positionwhere the guideline 440 is coincident with the shoulder edge 414 of theroad as a target in the parking space 413, the parking will becompleted. When the predictive traveling path curve 405 will possiblycome in contact with other vehicle or vehicles 415 and 416, the driverstops the backward movement for the series parking, and advances thevehicle 401. And if possible, the driver changes the direction of thevehicle 401 and tries to make the series parking.

FIG. 55 shows a control procedure in the embodiment of FIG. 54. Thecontrol procedure starts in a step a1. In a step a2, what executes theprocedure, or the DSP 420 of the parking assist ECU 406, judges whetheror not given series parking assist conditions hold. When the back lampSW signal 412 is in an on state, and the left winker lamp 409L or theright winker lamp 409R is turned on, the DSP judges that the seriesparking assist conditions hold. When the winker lamps 409L and 409R areboth turned on for the hazard lamp, the DSP may judge that the vehiclewill be series parked on and along the left side of the road as usual.If the series parking assist conditions hold, the DSP judges which thesides of the road is used for the series parking in a step a3. In thenext step a4, the DSP detects a target parking position 445 from theimage picked up by the camera unit 410. In a step a5, the DSP detects asteering angle of the steering wheel 407 of the vehicle 401 by anangular displacement signal output from the steering sensor 411. In astep a6, the DSP calculates a predictive traveling path curve 405depending on the steering angle. In a step a7, the DSP judges whether ornot the vehicle 401 may be moved from the present position to the targetparking position 445 by the steering operation. When possible, the DSPjudges a timing of turning the steering wheel reversely in a step a8.After the reverse turn of the steering wheel, a position of the guideline 440 which is indicative of the reverse turn timing on thepredictive traveling path curve 405 is determined and the resultant isadded to the image. In a step a10, the DSP judges whether or not theseries parking assist conditions hold, as in the step a2. When theseries parking assist conditions hold, the DSP returns to the step a5,and continues the parking assist in a similar manner.

The judgement of the step a7 as to whether or not the parking is allowedis made on the basis of the gathering of position data of the parkingspace 413 for the vehicle 401 at the start of parking, comparison of itwith the data of the guide line 440 when the vehicle 401 is put inparallel, which is stored in advance, and a minimum turn radius of thevehicle 401, and the like. When the step a7 judges that the parking isimpossible, the DSP displays a guide of “parking impossible” by theinformation display device 404 in a step all. When the step a2 or a10judges that the series parking assist conditions do not hold, or thatthe step all displays the parking-impossible guide, the DSP ends itsprocedural process at a step a12.

FIG. 56 shows an image for the series parking assist in 20th embodimentof the present invention. In the embodiment, like or equivalent portionsare designated by like reference numerals in FIG. 54. A parking space413 is located between other vehicles 415 and 416, and a target parkingposition 445 is detected. A predictive traveling path curve 405 isdisplayed, which is predictively generated on the basis of a steeringangle of the present steering wheel 407, and an ideal traveling pathcurve 446 ranging from the present vehicle position to the targetparking position 445 is displayed. Accordingly, the driver of thevehicle 401 can easily park the vehicle 401 in the target parkingposition 445 in a manner that he operates the steering wheel 407 so asto approximate the predictive traveling path curve 405 in shape to theideal traveling path curve 446.

FIG. 57 is a flow chart showing a control procedure by the DSP in theFIG. 56 embodiment. A procedural sequence from steps b1 to b6 isequivalent to that from the steps a1 to a6 in the FIG. 55 flow chart. Astep b7 is executed to generate an ideal traveling path curve 446 formoving the vehicle 401 from the present position of the vehicle 401 tothe target parking position 445 detected in the step b4. The idealtraveling path curve 446 is generated on the basis of position data ofthe target parking position 445 in the parking space 413, which gatheredat the present position of the vehicle 401 at the start of the parking,and a minimum turn radius of the vehicle, a speed of the backingvehicle, and the like. A step b8 is executed to check whether or not thegeneration of the ideal traveling path curve 446 ends. When thegeneration ends, a step b9 is executed to generate a parking assistimage in which the predictive traveling path curve 405 calculated in thestep b6 and the ideal traveling path curve 446 generated in the step b7are overlaid on the backward area image output from the camera unit 410,and to display the parking assist image on the screen of the informationdisplay device 404. A step b10, like the step b2, is executed to judgewhether or not the series parking assist condition holds. When thecondition holds, the control by the DSP returns to the step b5.Subsequently, a procedural sequence of the steps b5 to b10 is repeated.

An ideal traveling path curve 446 based on a steering angle isdisplayed, if it is within a certain deviation of it from the ideal one,without operating the steering wheel 407 along the ideal traveling pathcurve 446, and the vehicle may be parked at the target parking position445. When the step b8 judges that the generation of the ideal travelingpath curve 446 does not end yet, a step b11 is executed to display theguide of “parking impossible”. When the step b2 or b10 judges that theseries parking assist condition does not hold, or when the process ofthe step b11 ends, a step b12 is executed to end the parking assistoperation.

While in each of the above-mentioned embodiments, the camera unit 410 isused for taking a scene of the place behind the vehicle for the seriesparking, the camera unit 410 may also be used for an out-of-lanemonitoring purpose. In this case, the camera unit constantly recognizesthe white lines 403 and checks whether the running vehicle is within orout of the traffic lane. If the camera unit 410 is thus used for boththe purpose of monitoring the place behind the vehicle for the seriesparking assist and the purpose of monitoring the out-of-lane of thevehicle, the required number of cameras installed on the vehicle isreduced, leading to cost reduction.

In the vehicle parking assist device, an image of the area existing inthe vehicle advancing direction is displayed, and a vehicle predictivetraveling path predicted on the basis of a steering angle of thesteering wheel and guide lines prolonged from the lines defining thewidth of the vehicle body are overlaid on the image of the area existingin the vehicle advancing direction, whereby the drive assist isperformed. When the vehicle is backed for parking, the guide linesprolonged from the lines defining the width of the vehicle body areoverlaid on the image of the backward area. Therefore, the presentdirection of the vehicle is easily understood. Where white lines aredrawn for the parking frame, the vehicle may easily be parked inparallel to the parking frame in a manner that when the white lines areparallel to the guide lines, the steering wheel is returned to thecenter position.

Where displays of the predictive traveling path and/or guide lines areselectively presented, the drive assist is performed in the displaycondition best for the driving by the driver.

In the vehicle parking assist device, a brightness of a display of thepredictive traveling path and/or guide lines is varied in accordancewith a brightness of the environment. The drive assist is performed inan easy-to-see display condition.

In the vehicle drive assist device, the drive assist means varies acolor arrangement of a display of the predictive traveling path and/orguide lines in accordance with a color arrangement of the environment. Acolor arrangement of the display is changed to a color arrangement whichis easy for the driver to understand in accordance with a brightness anda color arrangement of the environment, whereby an effective driveassist is carried out.

In the vehicle drive assist device, the vehicle includes a brightnesssensor for sensing a brightness of the environment to select anillumination system, and the drive assist means varies a brightness or acolor arrangement of a display of the predictive traveling path and/orguide lines in accordance with a brightness sensed by the brightnesssensor:

Therefore, a display of the display means may be automatically changedto an easy-to-see display in accordance with a brightness and a colorarrangement of the environment.

In the invention, a color arrangement of a display of the predictivetraveling path and/or guide lines is varied in accordance with abackground color of an image picked up by the camera. With this feature,when the background color is bright, the predictive traveling path curveand the guide lines may be displayed using a dark color. When thebackground color is dark, the latter displayed may be bright in color.Therefore, an easy-to-understand drive assist may be presented to thedriver.

In the invention, the final display condition at the end of the parkingassist is stored, and then the display condition is used for the nextparking drive.

Therefore, the drive assist may be performed in the display conditionsuitable for the parking guide without setting anew, by the driver, anewthe best display condition every time the drive assist is performed.

An image output from the camera of a wide angle field is distorted. Inthe invention, said guide lines, when displayed, are curved inaccordance with a distortion of the image. Therefore, the guide linesexactly correspond to the lines respectively prolonged from the linesdefining the width of the vehicle body, whereby an effective driveassist is formed.

The width of the predictive traveling path curve and the guide lines arenot coincident with the width of an actual vehicle body at its bumperand its near portion because of the field angle of the camera has afield angle. Therefore, in the invention, the non-coincident portion isnot displayed, and the driving for this portion depends on the driver'sjudgement, whereby an unnatural display is eliminated.

In the vehicle drive assist device, the drive assist means displays thepredictive traveling path and/or guide lines such that the portionsthereof near the vehicle are broadened. Therefore, the vehicle width isnaturally contiguous to the predictive traveling path and the guidelines in the display. And a safety is secured.

In the invention, a line indicative of a stop position of the vehicle isdisplayed at a position on the predictive traveling path or the guidelines, which is located near the vehicle. Therefore, the line may beusefully utilized as such a mark that when the moving vehicle framereaches the stop line, the vehicle is stopped.

In the invention, the guide lines are displayed when the driver operatesthe brake. In other words, when the braking operation is not performed,the guide lines are not displayed, thereby eliminating a complexity ofthe drive assist image. Therefore, such a drive assist that whennecessary, the guide lines are displayed by the braking operation andthe steering wheel is returned to the center position, is effectivelyperformed.

In the invention, the guide lines are displayed by dotted lines, andhence are made unnoticeable. Therefore, the predictive traveling path iseasy to see, and the drive assist is performed in an easy-to-understandfashion.

In the invention, a length of the guide lines is varied in accordancewith a distance ranging to an obstructive object detected by theobstacle sensor, such as a back sonar. Therefore, the length of theguide line corresponds to the remaining driving distance. A drivefeeling resembles an actual driving feeling, and a driver assist whichis easy for the driver to understand is provided.

In the invention, the width of the guide lines, when displayed, is widerthan the width of the vehicle. Therefore, such a drive assist that whenthe guide line lies on one side of the parking frame, the driver returnsthe steering wheel to the center position is readily performed if thewidth of the guide lines is wider than that of the actual vehicle width.

In the vehicle parking assist devices, an image of a backward area ofthe vehicle is picked up, and an effective drive assist is performed.When the transmission is set to the reverse gear mode, a display by thedisplay means changes to inform the driver of the transmission being ina reverse gear mode. The display change informs the driver that thetransmission is shifted to a reverse gear position, and the vehicle willback, in an easy-to-understand manner. Accordingly, the driver nevermakes a mistake of the fact that the advancing direction of the vehicleis a backward movement.

In the invention, when an obstructive object is detected by the obstaclesensor such as a back sonar or corner sensors, a display by the displaymeans which displays a backward area image of the vehicle is changed toinform the driver of the presence of the obstructive object in aneasy-to-understand manner, thereby preventing the vehicle from touchingthe obstructive object.

In the invention, the drive assist information is displayed in the formof character telop. Therefore, the driver is reliably informed of thecontents of the display.

In the vehicle drive assist device, the drive assist means displaysinformation of the reverse gear position and obstacle detection in theform of a color change of the entire display. The color change of theentire display reliably calls a driver's attention.

In the vehicle parking assist devices, a telop for urging the driver tomake a safety check by the eye are displayed on the display forassisting the drive assist by use of an image picked up by the camera.Therefore, driver's attention is attracted to the front and side areasof the vehicle, which are not displayed on the display screen.

When the vehicle backs for parking, the vehicle parking assist deviceinforms the driver of the end of the parking operation while beinginterconnected in operation with an obstacle sensor, such as a backsonar. Therefore, the driver can readily check the detection result ofthe obstacle sensor by the display means.

In the invention, when the vehicle is moving backward, a speed of thevehicle is limited to the preset upper limit of the vehicle speed orlower. Therefore, collision of the vehicle when it is moved backward isprevented or shock when the collision occurs is lessened.

In the invention, when the obstacle sensor detects an obstacle such as aback soar, the drive assist means limits a speed of the vehicle to thepreset upper limit of the vehicle speed or lower. Therefore, collisionof the vehicle when it is moved backward is prevented or shock when thecollision occurs is lessened.

When the vehicle is moving backward, the drive assist means changes anillumination direction of a lamp illuminating a backward area of thevehicle, thereby effecting the drive assist. Therefore, when the parkinglot is dark at night, the image picked up by the camera is increased inbrightness, thereby increasing the visibility.

In the invention, the lamp is optionally attached to the device.Therefore, an appropriate illumination is performed in accordance with acoverage by the camera.

In the vehicle drive assist device, the vehicle is equipped with acar-carried device including a navigation device and/or audio device,and further it comprises select means for causing the display means toselectively display the image from the camera and the image from thedrive assist means or the image from the car-carried device. Therefore,the driver selects the drive assist or the utilization of thecar-carried device in accordance with his necessity.

In the vehicle parking assist devices, a scene of the area in theadvancing direction of the vehicle is picked up by the camera, and adrive assist useful for the parking drive is performed. Particularly, anormal parking mode or a series parking mode may be selected. Therefore,the drive assist suitable for the parking mode may be effectivelyperformed.

The parking mode is automatically changed to a series parking mode whenthe turning on of a winker lamp or a hazard lamp and the shifting of thetransmission to the reverse gear mode are simultaneously performed. Aproper drive assist is performed without a manual operation by thedriver to change the parking mode change.

In the invention, the drive assist means judges whether a parkingposition in which the vehicle is to be parked is in a parking lot or ona road, on the basis of the present position of the vehicle derived fromthe navigation device. When the parking position is in the parking lot,the normal parking mode is automatically selected, and when the parkingposition is on the road, the series parking mode is automaticallyselected, whereby a proper drive assist is performed.

Since the normal parking mode or the series parking mode is selecteddepending on a change of steering angle of the steering wheel forsteering the vehicle, the parking mode may be automatically changed tothe series parking mode by a special steering operation, such as thereverse turn of the steering wheel, which is used for the seriesparking.

In the invention, an effective drive assist is carried out by use ofimages output from a plurality of cameras are disposed such that thephotographing areas covered by the cameras are shifted so as to pick upa scene around the vehicle. A display screen of the display means issegmented into plurality of segmental display areas in accordance withthe arrangement of the cameras. The images by the plurality of camerasare simultaneously displayed on those segmental display areas.Therefore, the driver may look a plurality of images on the same screenof the display means, whereby an effective drive assist is performed.

In the invention, a display on the screen is changed in accordance witha drive condition. Accordingly, a display of visual information which iseasily understood by the driver may be presented through an automaticchanging of a screen segmentation such that in the normal running, twoimages of the front left and front right areas of the vehicle aredisplayed, and in the backward running, three images of the front leftand front right areas, and the backward area are displayed.

Also in the invention, a vehicle is displayed in a simplified form on adisplay of the images coming from the plurality of cameras. Therefore, arelationship of the vehicle relative to the images from the plurality ofcameras is displayed in an easy-to-understand fashion.

In the vehicle drive assist device, a display position of the simplifieddisplayed vehicle is moved according to an image from the camera.Therefore, the driver easily recognizes which camera picked up the imagebeing displayed from the display.

When the vehicle is displayed in a simplified form, the directions andphotographing areas covered by the cameras are also displayed.Therefore, the driver easily recognizes which camera picked up the imagebeing displayed from the display.

In the vehicle drive assist device, the images from the plurality ofcameras and an image from the car-carried device, such as a navigationdevice and/or audio device, are displayed on the segmental displayareas.

Therefore, the driver may look both the image by each camera and theimage by the navigation device or the audio device on the same displayscreen.

In the invention, the images by the plurality of cameras are changed oneto another at predetermined time intervals, to thereby eliminate themanual work of the driver to change the images.

In the invention, the wipe is used for gradually changing the imagesfrom the plurality of cameras one to another. Therefore, there is nochance that the image is abruptly changed from one image to another.

In the vehicle drive assist device, a scene of a side area of thevehicle is displayed while being interconnected with an operation of thedirection indicator. Therefore, an image taken in the vehicle advancingdirection is automatically displayed, whereby the vehicle parking assistdevice effectively assists the driver to make a safety check.

In the invention, the images are changed one to another in accordancewith a steering angle of turn of a steering wheel. Therefore, the imageof the area existing in an actual vehicle advancing direction isautomatically displayed, and the device assists the driver safely andeffectively.

In the invention, the screen segmenting condition is changed inaccordance with a steering angle of the steering wheel. Therefore, theimage of the area located in the advancing direction is displayed on awide display screen.

In the invention, when an obstructive object is detected by the obstaclesensor, e.g., a corner sensor, an obstacle sensing direction isdisplayed. The vehicle parking assist device calls driver's attention tothe presence of the obstacle.

An image located in the obstacle sensing direction is displayed, it maybe checked on the screen.

In the vehicle drive assist device, when an obstacle is detected, thedrive assist means causes the display means to shift a display positionof an image output from the camera which picks up the sensing directionof the obstacle sensor, to the center of the screen of the display meansor to expand the display.

Therefore, the resultant display reliably attracts the driver'sattention to the presence of the obstructive object and its state.

In the invention, an alarm is overlaid on a display of the sensingdirection of the obstacle sensor. Therefore, the alarm reliably turnsthe driver's one's attention to the obstructive object.

In the vehicle drive assist device, when the images of the camerasadjacently disposed are simultaneously displayed, those images arecontinuously connected together into a single image. With the composedimage, the image covering an expanse of area, which is easy for thedriver to understand, is provided to the driver, to thereby effect aneffective drive assist.

In the vehicle drive assist device, the camera are disposed so as totake he scenes of the front, front left, front right, back left and backright areas of the vehicle. Therefore, the driver can make a safetycheck on the entire circumferential area around the vehicle.

Further, the image of the entire circumferential area around the vehiclemay effectively be used in accordance with an arrangement of thecameras.

Additionally, a necessary image is selected from those output from thecameras disposed around the vehicle, and is displayed in detail.

The images output from the cameras having the photographing areassuccessively disposed are scrolled into a single image. Therefore, thedriver is able to readily check the image of a broad area around thevehicle in a natural fashion.

The driver can change a display condition of a desired image in simpleand quick manner by use of the touch panel.

Further, the driver causes the display means to display a desired imagefor an elongated time in simple and quick manner.

In the vehicle parking assist device, an effective drive assist isperformed by use of a camera for picking up a frontward scene of thevehicle. An image of the frontward scene is stored into the storingmeans, thereby acquiring information for the drive assist. Accordingly,the image by the camera may also be used as traffic congestioninformation, whereby it may be effectively used for grasping trafficconditions around the vehicle.

Since the frontward image picked up by the camera is displayed by thedisplay means, it may be used for checking the frontward scene.Therefore, use of the single camera reduces the cost when comparing witha case where cameras are respectively provided for picking up thefrontward and backward scenes.

In the invention, a vehicle forcibly running into a place in front of aself vehicle is detected by processing the image coming from saidcamera. Therefore, the camera may be used for detecting such a vehicle,and even if such a vehicle appears, the driver drives the vehicle insafe.

In the vehicle parking assist device, a camera is attached to the top ofa vehicle and picks up a scene around a vehicle as viewed from the topof the vehicle. Display means displays the image picked up by thecamera. A limit of a vehicle height is detected from the image picked upby the camera. Therefore, the driver can readily check whether or notthe vehicle may pass under an elevated bridge, for example.

In the vehicle parking assist device, in accordance with the movingdirection of the vehicle, in n a backward running, the display meansdisplays more brightly an image of the backward area than an image ofthe forward area. Therefore, the driver can watch the backward area indetail when backing the vehicle.

In the vehicle parking assist device, an image of the backward area fromthe camera is displayed reflecting an image on the back mirror.Accordingly, the image of the backward area is displayed in a displaymode familiar to the driver. The drive assist is effectively carriedout.

In the vehicle drive assist device, the drive assist means modifies abackward area image displayed in the display frame for the back mirrorsinto an image having a field angle equal to that of the back mirror.

Therefore, the driver looks the backward area image as if he looks theback mirror, whereby an effective drive assist is performed.

When a parking assist is carried out in a state that a predictivetraveling path curve, which is calculated depending on a steering angleis overlaid on a display of the image which is picked up in the vehicleadvancing direction, and a 3-three dimensional image containing thevehicle height is additionally displayed along the predictive travelingpath curve, a display of the 3-dimensional image is changed at aposition near an obstructive object detected by the obstacle sensor,from that at other positions. Therefore, it is easy to check apossibility that the vehicle will touch an obstructive object near thetraveling path of the vehicle, and a touching position when it touchesthe obstructive object. Accordingly, the vehicle drive assist device canassist the driver in his driving operation such that the touching of hiscar with the obstructive object is avoided and the driving operation iseasy.

In the invention, the drive assist means displays the 3-dimensionalinformation image in the form of planes each having a predeterminedshape which are arrayed at fixed distances along the predictivetraveling path curve. Therefore, the driver readily checks from thedisplayed image a space occupied by the vehicle when it advances and a3-dimensional touch possibility of the vehicle with an obstructiveobject existing along the traveling path.

In the invention, the 3-dimensional image containing the vehicle heightmoves as a plane of a predetermined shape, which represents a height ofthe vehicle, along the predictive traveling path curve. Therefore, thedriver readily may check a possibility that the vehicle will touch anobstructive object located near the predictive traveling path curve anda touching position when it touches the object.

In the invention, a plane of a predetermined shape, which represents aheight of the vehicle, is moved along the predictive traveling pathcurve, and is stopped at a position close to the obstructive object.Therefore, the driver 3-dimensionally may check in detail a possibilityof touching of the vehicle with the obstructive object.

In the invention, the drive assist changes a color of the 3-dimensionalinformation image at a position which is close to a position where theobstructive object is detected. Therefore, the driver readily may check,from a change of the color, a possibility that the vehicle will touch anobstructive object located near the predictive traveling path curve anda touching position when it touches the object.

In the invention, a display is presented such that a simulation image ofthe self vehicle moves in the image picked up by the camera. Therefore,the driver checks a possibility touching of the vehicle with anobstructive object in a simulation manner.

In parking the vehicle, a target parking position is detected, asteering angle necessary for the advancing of the car to the targetparking position is calculated, a drive assist for parking is carriedout with such a guide that a steering angle is approximate to thecalculated one. Since the driver is supported so as to be able to moresafely drive the vehicle, the driver can easily park the vehicle into anarrow parking space or a congested parking lot.

When the vehicle, which advances for its parking, cannot advance to atarget parking position by a steering operation within a range ofsteering angles within which the steering wheel may be turned, or whenit is judged that the vehicle will touch an obstructive object duringits advancing, the driver is given an instruction to reverse anadvancing direction of the vehicle. Accordingly, the drive assist caninform the driver of the parking being possible, and the necessity andtiming of the reverse turn of the steering wheel. It never happens thatthe driver tries to advance the vehicle to a parking space which rejectsits parking, and causes a trouble of touching of his vehicle withanother vehicle. In this respect, the driver can safely drive thevehicle.

In a case that a parking stopper exists in the parking space, it isautomatically detected and the driver is informed of the presence of theparking stopper. Therefore, the driver may drive the vehicle for parkingwhile referring to the stopper.

A voice guide makes it for the driver to park the vehicle. In this case,the drive assist is easy to understand since the visual sensation of thedriver is not directly utilized.

The drive assist of the invention gives the driver warning of watchingvehicle environments, e.g., the front and side of the vehicle, when heparks the vehicle. Therefore, safe drive is ensured by turning hisattention to the vehicle environmental conditions.

The drive assist for parking of the invention displays a parking guideimage overlaid on the image of an area existing in the advancingdirection, which is picked up by the camera. This parking guide displayis easy to understand.

In the present invention, the guide lines prolonged from the linesdefining the width of the vehicle and distance lines are overlaid on thebackward area image, thereby providing a parking assist which is easyfor the driver to understand.

The predictive traveling path curve 305 overlaid on the backward areaimage is adjusted in accordance with a distance to a stop position.Therefore, a path along which the vehicle runs till it parks isdisplayed in the backward area image such that the driver easilyunderstands the path. Accordingly, the vehicle parking assist device ofthe invention appropriately assists the driver in his parking.

In the invention, a roof predictive traveling path curve of the selfvehicle is displayed overlaid on the backward area image. Even if aportion of the self vehicle which will be a dead space to the driverbecause of the presence of another vehicle adjacent to the self vehicle,is present in the backward area image, the driver adjusts the predictivetraveling path curve so as to avoid colliding of the self vehicle withthe roof of the vehicle, and moves back the vehicle for parking, wherebyan effective parking assist is performed.

Even if the predictive traveling path curve of the self vehicle isshaded by another vehicle parked in a parking space adjacent to aparking space in which the self vehicle is to be parked in the backwardarea image, a projective line of the other vehicle to the ground isdisplayed, and the vehicle parking assist device assists the driver inhis parking by use of the predictive traveling path curve and theprojective line to the ground. Therefore, even in an area forming a deadspace in the image, it is possible to reliably check a positionalrelationship of the predictive traveling path curve of the self vehicleranging to the parking space to other vehicles.

The vehicle parking assist device of the invention displays a layout ofthe self vehicle and other vehicles in the parking areas when viewedfrom the top by use of a plane model. Therefore, the driver can easilygrasp a positional relationship of the self vehicle, a parking space,and other vehicles parked in other parking spaces, and an orientationand moving direction of the self vehicle.

In the vehicle parking assist device, a parking guide by the plane modelor a parking guide by the backward area image is selected in use. Forexample, a parking assist may appropriately be selected according toparking space conditions and the like.

Further, in the vehicle parking assist device of the invention, afterthe movement of the vehicle is switched from a backward movement to aforward movement, the displaying of a parking assist image of thebackward area is continued for a fixed time. Therefore, the driver caneasily check the backward area of the vehicle for advancing the vehicleor reversely turning the steering wheel.

The present invention enables the driver to easily make a series parkingwhen he moves back the vehicle and parks it in a space between other twoadjacent vehicles parked in row since in such a situation, a predictivetraveling path curve based on an steering angle is overlaid on an imageof a place behind his vehicle on the screen of an information displaydevice, and the display is added with guide information for a steeringoperation that is performed at the timing of the steering wheel reverseturn for the series parking.

In the invention, the timing of the steering wheel reverse turn isdepicted in the form of a predictive traveling path curve. Accordingly,the driver can easily make a series parking by operating the steeringwheel according to a relationship between an oblique line and a parkingtarget.

In the invention, the series parking assist automatically commences bymerely performing an operation necessary for the driving operation.Therefore, there is eliminated cumbersome switch operations whichotherwise would be unavoidable in the series parking.

In the invention, judgement is made as to whether a place for the seriesparking is on the right side or the left side of the vehicle on thebasis of the combination of lighting conditions of the hazard lampand/or the winker lamp. Therefore, the series parking assist isautomatically performed for a proper side of the road.

In the present invention, the image by the camera may also be used forthe lane-off monitoring. Therefore, there is no need of providingcameras respectively for the series parking monitoring and the lane-offmonitoring. A single camera may be used for both the purposes, resultingcost reduction.

In the invention, whether or not the series parking is possible isjudged before the parking operation starts. If the series parking isimpossible, the driver changes the direction of the vehicle travelingpath, and will find another parking space.

1. A vehicle drive assist device adapted to be mounted on a vehicle,comprising: a first camera installed at a rear side of the vehicle topick up a first image; a second camera installed at a right side of thevehicle to pick up a second image; a third camera installed at a leftside of the vehicle to pick up a third image; a display having a screenon which at least one of the images picked up by the cameras isdisplayed; and a drive assist controller operable to divide the screeninto a plurality of display areas, and display the respective images onthe respective display areas, wherein the drive assist controllerdisplays the first image, the second image and the third imagesimultaneously on an entire area of the screen; wherein the drive assistcontroller displays a simplified figure of the vehicle, which indicatesan entire appearance of the vehicle as viewed from above at the centerof the entire area of the screen; and the drive assist controllerdisplay respective photographing directions of the first to thirdcameras on the respective display areas so as to superimpose onto therespective first to third images.
 2. The vehicle drive assist deviceaccording to claim 1, further comprising a drive condition detectoroperable to detect a drive condition of the vehicle, wherein the driveassist controller selectively changes an image to be displayed on thescreen, which is picked up by one of the cameras in accordance with thedrive condition detected by the drive condition detector.
 3. The vehicledrive assist device according to claim 1, wherein the drive assistcontroller moves a display position of the simplified figure of thevehicle on the screen according to at least one of the images.
 4. Thevehicle drive assist device according to claim 1, wherein the driveassist controller controls the display to display a direction and aphotographing area of each of the cameras when the simplified figure isdisplayed.
 5. The vehicle drive assist device as set forth in claim 1,wherein the vehicle on which the drive assist device is mounted has acar-carried device including at least one of a navigation device andaudio device; and wherein the images from the cameras and an image fromthe car-carried device are displayed on the display areas, respectively.6. The vehicle drive assist device according to claim 1, wherein whenone of the images which is picked up by one of the cameras is displayedon one of the display areas, the drive assist controller changes the oneof the image to another image which is picked up by another of thecameras at a predetermined time interval.
 7. The vehicle drive assistdevice according to claim 6, wherein the drive assist controller uses awipe to change the images.
 8. The vehicle drive assist device accordingto claim 1, wherein one of the cameras is directed to a side of thevehicle for picking up an image of a side area of the vehicle; andwherein the drive assist controller controls the display to display theimage of the side are being interconnected with an operation of adirection indicator mounted on the vehicle.
 9. The vehicle drive assistdevice according to claim 1, wherein the drive assist controller changesan image to be displayed on the screen, which is picked up by one of thecameras in accordance with a steering angle of a steering wheel mountedon the vehicle.
 10. The vehicle drive assist device according to claim1, wherein when adjacent images of the cameras adjacently disposed aresimultaneously displayed, the drive assist controller adjusts thedisplay areas and positions of the adjacent images to connect theadjacent images continuously with each other.
 11. The vehicle driveassist device according to claim 1, wherein the plurality of cameras aredisposed to take scenes of the front, front left, front right, back orrear left and back or rear right areas of the vehicle.
 12. The vehicledrive assist device according to claim 11, wherein the drive assistcontroller divides the display screen of the display into upper andlower display areas, and causes the display to display an image in frontof the vehicle on the central region of the upper portion of the screen,images of front left and front right images on the left and rightdisplay areas of the upper display area, and an image of the rear areaof the vehicle on the central area of the lower display area of thedisplay screen, images of the rear left and rear right on the left andright display areas of the lower display area of the display screen ofthe display.
 13. The vehicle drive assist device according to claim 11,wherein the drive assist controller causes the display to display animage selected from the images picked up by the plurality of cameras.14. The vehicle drive assist device according to claim 11, wherein thedrive assist controller causes the display the images output from theplurality of cameras having the photographing areas successively inscrolling manner.
 15. The vehicle drive assist device according to claim1, wherein the display includes a touch panel, and the drive assistcontroller changes an image display condition according to an input tothe touch panel.
 16. The vehicle drive assist device according to claim15, wherein according to the input to the touch panel, the drive assistcontroller enlarges and displays an image at an input position at apredetermined time, and after the predetermined time elapses, the driveassist controller put back the image enlarged and displays the image onthe display screen.
 17. The vehicle drive assist device according toclaim 1, wherein each of the display areas displays each of the imagessuch that a location of each of the display areas relative to thedisplay screen corresponds to a direction of each of the photographingareas relative to the vehicle.
 18. The vehicle drive assist deviceaccording to claim 1, wherein each of the cameras faces to a differentdirection relative to the vehicle; wherein each of the display areasdisplays each of the images such that a location of each of the displayareas relative to the simplified figure of the vehicle corresponds tothe direction of each camera relative to the vehicle.
 19. A vehicledrive assist device adapted to be mounted on a vehicle, comprising: aplurality of cameras covering different photographing areas from eachother, to pick up images around the vehicle; a display having a screenon which at least one of the images picked up by the cameras isdisplayed; and a drive assist controller operable to divide the screeninto a plurality of display areas, and display the respective images onthe respective display areas, wherein the drive assist controllerdisplays a simplified figure of the vehicle on the screen with theimages picked up by the cameras, wherein the drive assist controllerchanges an image to be displayed on the screen, which is picked up byone of the cameras in accordance with a steering angle of a steeringwheel mounted on the vehicle, and wherein the drive assist controllerchanges a condition of dividing the screen in accordance with thesteering angle of the steering wheel.
 20. The vehicle drive assistdevice according to claim 1, further comprising an obstacle sensor fordetecting presence or absence of an obstacle on a vehicle travelingpath, wherein when the obstacle sensor detects the obstacle, the driveassist controller causes the display to display a sensing direction ofthe obstacle sensor.
 21. The vehicle drive assist device according toclaim 20, wherein when the obstacle sensor detects an obstacle, thedrive assist controller causes the display to display an image fromcamera picking up a sensing direction of the obstacle sensor.
 22. Thevehicle drive assist device according to claim 20, wherein when theobstacle sensor detects the obstacle, the drive assist controller causesthe display to shift a display position of the image from the camerapicking up the sensing direction of the obstacle sensor, to the centerof the screen of the display or to expand the display.
 23. The vehicledrive assist device according to claim 20, wherein the drive assistcontroller causes the display to overlay an alarm on a display of thesensing direction of the obstacle sensor.